2023-02-12 04:00:43,844 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-12 04:00:44,056 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8 2023-02-12 04:00:44,072 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-12 04:00:44,073 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd, deleteOnExit=true 2023-02-12 04:00:44,073 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-12 04:00:44,074 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/test.cache.data in system properties and HBase conf 2023-02-12 04:00:44,074 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/hadoop.tmp.dir in system properties and HBase conf 2023-02-12 04:00:44,075 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/hadoop.log.dir in system properties and HBase conf 2023-02-12 04:00:44,075 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-12 04:00:44,076 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-12 04:00:44,076 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-12 04:00:44,175 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-12 04:00:44,511 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-12 04:00:44,515 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-12 04:00:44,516 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-12 04:00:44,516 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-12 04:00:44,516 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-12 04:00:44,516 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-12 04:00:44,517 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-12 04:00:44,517 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-12 04:00:44,517 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-12 04:00:44,518 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-12 04:00:44,518 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/nfs.dump.dir in system properties and HBase conf 2023-02-12 04:00:44,519 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/java.io.tmpdir in system properties and HBase conf 2023-02-12 04:00:44,519 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-12 04:00:44,519 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-12 04:00:44,520 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-12 04:00:44,979 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-12 04:00:44,982 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-12 04:00:45,222 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-12 04:00:45,383 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-12 04:00:45,400 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-12 04:00:45,435 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-12 04:00:45,466 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/java.io.tmpdir/Jetty_localhost_localdomain_34799_hdfs____.kkm6c0/webapp 2023-02-12 04:00:45,601 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:34799 2023-02-12 04:00:45,611 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-12 04:00:45,611 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-12 04:00:45,975 WARN [Listener at localhost.localdomain/37867] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-12 04:00:46,086 WARN [Listener at localhost.localdomain/37867] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-12 04:00:46,105 WARN [Listener at localhost.localdomain/37867] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-12 04:00:46,111 INFO [Listener at localhost.localdomain/37867] log.Slf4jLog(67): jetty-6.1.26 2023-02-12 04:00:46,116 INFO [Listener at localhost.localdomain/37867] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/java.io.tmpdir/Jetty_localhost_46621_datanode____jgtsii/webapp 2023-02-12 04:00:46,197 INFO [Listener at localhost.localdomain/37867] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46621 2023-02-12 04:00:46,469 WARN [Listener at localhost.localdomain/41099] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-12 04:00:46,480 WARN [Listener at localhost.localdomain/41099] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-12 04:00:46,486 WARN [Listener at localhost.localdomain/41099] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-12 04:00:46,489 INFO [Listener at localhost.localdomain/41099] log.Slf4jLog(67): jetty-6.1.26 2023-02-12 04:00:46,495 INFO [Listener at localhost.localdomain/41099] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/java.io.tmpdir/Jetty_localhost_36137_datanode____.mzkavb/webapp 2023-02-12 04:00:46,581 INFO [Listener at localhost.localdomain/41099] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36137 2023-02-12 04:00:46,589 WARN [Listener at localhost.localdomain/41885] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-12 04:00:46,606 WARN [Listener at localhost.localdomain/41885] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-12 04:00:46,610 WARN [Listener at localhost.localdomain/41885] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-12 04:00:46,612 INFO [Listener at localhost.localdomain/41885] log.Slf4jLog(67): jetty-6.1.26 2023-02-12 04:00:46,617 INFO [Listener at localhost.localdomain/41885] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/java.io.tmpdir/Jetty_localhost_33531_datanode____.avkpt2/webapp 2023-02-12 04:00:46,727 INFO [Listener at localhost.localdomain/41885] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33531 2023-02-12 04:00:46,745 WARN [Listener at localhost.localdomain/35207] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-12 04:00:46,943 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xcedd297c1250319c: Processing first storage report for DS-db7ce3c1-4d75-4e76-bb51-66241fa898fb from datanode 484548f8-2ed9-47b7-81ed-a4821ba20939 2023-02-12 04:00:46,945 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xcedd297c1250319c: from storage DS-db7ce3c1-4d75-4e76-bb51-66241fa898fb node DatanodeRegistration(127.0.0.1:33265, datanodeUuid=484548f8-2ed9-47b7-81ed-a4821ba20939, infoPort=32853, infoSecurePort=0, ipcPort=35207, storageInfo=lv=-57;cid=testClusterID;nsid=1739323266;c=1676174445046), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-12 04:00:46,945 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x423a9811d5451b4a: Processing first storage report for DS-1e6f0745-09f2-486f-9b8d-507ec526706c from datanode 6bee6fa0-f002-488e-898d-32eb09388f7e 2023-02-12 04:00:46,946 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x423a9811d5451b4a: from storage DS-1e6f0745-09f2-486f-9b8d-507ec526706c node DatanodeRegistration(127.0.0.1:46703, datanodeUuid=6bee6fa0-f002-488e-898d-32eb09388f7e, infoPort=35875, infoSecurePort=0, ipcPort=41885, storageInfo=lv=-57;cid=testClusterID;nsid=1739323266;c=1676174445046), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-12 04:00:46,946 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa24834dab7420ea: Processing first storage report for DS-4e98e139-83ef-42ee-bcd5-93432335188f from datanode ba999208-985a-462e-b295-ddee73c8b42c 2023-02-12 04:00:46,946 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa24834dab7420ea: from storage DS-4e98e139-83ef-42ee-bcd5-93432335188f node DatanodeRegistration(127.0.0.1:41527, datanodeUuid=ba999208-985a-462e-b295-ddee73c8b42c, infoPort=44727, infoSecurePort=0, ipcPort=41099, storageInfo=lv=-57;cid=testClusterID;nsid=1739323266;c=1676174445046), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-12 04:00:46,946 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xcedd297c1250319c: Processing first storage report for DS-f82cab87-7c24-4ea6-a3f4-521bc8b0e410 from datanode 484548f8-2ed9-47b7-81ed-a4821ba20939 2023-02-12 04:00:46,946 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xcedd297c1250319c: from storage DS-f82cab87-7c24-4ea6-a3f4-521bc8b0e410 node DatanodeRegistration(127.0.0.1:33265, datanodeUuid=484548f8-2ed9-47b7-81ed-a4821ba20939, infoPort=32853, infoSecurePort=0, ipcPort=35207, storageInfo=lv=-57;cid=testClusterID;nsid=1739323266;c=1676174445046), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-12 04:00:46,947 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x423a9811d5451b4a: Processing first storage report for DS-5fc21c4a-aabc-4efe-9ee9-da853fe3ed8d from datanode 6bee6fa0-f002-488e-898d-32eb09388f7e 2023-02-12 04:00:46,947 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x423a9811d5451b4a: from storage DS-5fc21c4a-aabc-4efe-9ee9-da853fe3ed8d node DatanodeRegistration(127.0.0.1:46703, datanodeUuid=6bee6fa0-f002-488e-898d-32eb09388f7e, infoPort=35875, infoSecurePort=0, ipcPort=41885, storageInfo=lv=-57;cid=testClusterID;nsid=1739323266;c=1676174445046), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-12 04:00:46,947 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa24834dab7420ea: Processing first storage report for DS-4fc41d9b-92a3-420c-b7e2-52c836ecb9ca from datanode ba999208-985a-462e-b295-ddee73c8b42c 2023-02-12 04:00:46,947 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa24834dab7420ea: from storage DS-4fc41d9b-92a3-420c-b7e2-52c836ecb9ca node DatanodeRegistration(127.0.0.1:41527, datanodeUuid=ba999208-985a-462e-b295-ddee73c8b42c, infoPort=44727, infoSecurePort=0, ipcPort=41099, storageInfo=lv=-57;cid=testClusterID;nsid=1739323266;c=1676174445046), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-12 04:00:47,284 DEBUG [Listener at localhost.localdomain/35207] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8 2023-02-12 04:00:47,352 INFO [Listener at localhost.localdomain/35207] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd/zookeeper_0, clientPort=56679, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-12 04:00:47,368 INFO [Listener at localhost.localdomain/35207] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=56679 2023-02-12 04:00:47,378 INFO [Listener at localhost.localdomain/35207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 04:00:47,381 INFO [Listener at localhost.localdomain/35207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 04:00:48,023 INFO [Listener at localhost.localdomain/35207] util.FSUtils(479): Created version file at hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2 with version=8 2023-02-12 04:00:48,023 INFO [Listener at localhost.localdomain/35207] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/hbase-staging 2023-02-12 04:00:48,298 INFO [Listener at localhost.localdomain/35207] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-12 04:00:48,674 INFO [Listener at localhost.localdomain/35207] client.ConnectionUtils(127): master/jenkins-hbase20:0 server-side Connection retries=45 2023-02-12 04:00:48,700 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-12 04:00:48,701 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-12 04:00:48,701 INFO [Listener at localhost.localdomain/35207] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-12 04:00:48,701 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-12 04:00:48,701 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-12 04:00:48,824 INFO [Listener at localhost.localdomain/35207] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-12 04:00:48,886 DEBUG [Listener at localhost.localdomain/35207] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-12 04:00:48,970 INFO [Listener at localhost.localdomain/35207] ipc.NettyRpcServer(120): Bind to /148.251.75.209:41569 2023-02-12 04:00:48,979 INFO [Listener at localhost.localdomain/35207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 04:00:48,980 INFO [Listener at localhost.localdomain/35207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 04:00:48,997 INFO [Listener at localhost.localdomain/35207] zookeeper.RecoverableZooKeeper(93): Process identifier=master:41569 connecting to ZooKeeper ensemble=127.0.0.1:56679 2023-02-12 04:00:49,030 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:415690x0, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-12 04:00:49,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:41569-0x10154d814950000 connected 2023-02-12 04:00:49,058 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-12 04:00:49,059 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-12 04:00:49,063 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-12 04:00:49,070 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41569 2023-02-12 04:00:49,070 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41569 2023-02-12 04:00:49,070 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41569 2023-02-12 04:00:49,071 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41569 2023-02-12 04:00:49,071 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41569 2023-02-12 04:00:49,076 INFO [Listener at localhost.localdomain/35207] master.HMaster(439): hbase.rootdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2, hbase.cluster.distributed=false 2023-02-12 04:00:49,140 INFO [Listener at localhost.localdomain/35207] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-02-12 04:00:49,140 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-12 04:00:49,140 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-12 04:00:49,140 INFO [Listener at localhost.localdomain/35207] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-12 04:00:49,141 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-12 04:00:49,141 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-12 04:00:49,145 INFO [Listener at localhost.localdomain/35207] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-12 04:00:49,149 INFO [Listener at localhost.localdomain/35207] ipc.NettyRpcServer(120): Bind to /148.251.75.209:40151 2023-02-12 04:00:49,151 INFO [Listener at localhost.localdomain/35207] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-12 04:00:49,157 DEBUG [Listener at localhost.localdomain/35207] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-12 04:00:49,158 INFO [Listener at localhost.localdomain/35207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 04:00:49,160 INFO [Listener at localhost.localdomain/35207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 04:00:49,162 INFO [Listener at localhost.localdomain/35207] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40151 connecting to ZooKeeper ensemble=127.0.0.1:56679 2023-02-12 04:00:49,165 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:401510x0, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-12 04:00:49,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:40151-0x10154d814950001 connected 2023-02-12 04:00:49,166 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-12 04:00:49,168 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-12 04:00:49,169 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-12 04:00:49,169 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40151 2023-02-12 04:00:49,171 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40151 2023-02-12 04:00:49,171 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40151 2023-02-12 04:00:49,174 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40151 2023-02-12 04:00:49,174 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40151 2023-02-12 04:00:49,185 INFO [Listener at localhost.localdomain/35207] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-02-12 04:00:49,185 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-12 04:00:49,185 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-12 04:00:49,186 INFO [Listener at localhost.localdomain/35207] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-12 04:00:49,186 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-12 04:00:49,186 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-12 04:00:49,186 INFO [Listener at localhost.localdomain/35207] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-12 04:00:49,188 INFO [Listener at localhost.localdomain/35207] ipc.NettyRpcServer(120): Bind to /148.251.75.209:43977 2023-02-12 04:00:49,189 INFO [Listener at localhost.localdomain/35207] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-12 04:00:49,191 DEBUG [Listener at localhost.localdomain/35207] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-12 04:00:49,191 INFO [Listener at localhost.localdomain/35207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 04:00:49,194 INFO [Listener at localhost.localdomain/35207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 04:00:49,197 INFO [Listener at localhost.localdomain/35207] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43977 connecting to ZooKeeper ensemble=127.0.0.1:56679 2023-02-12 04:00:49,201 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:439770x0, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-12 04:00:49,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43977-0x10154d814950002 connected 2023-02-12 04:00:49,203 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-12 04:00:49,203 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-12 04:00:49,204 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-12 04:00:49,207 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43977 2023-02-12 04:00:49,207 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43977 2023-02-12 04:00:49,207 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43977 2023-02-12 04:00:49,208 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43977 2023-02-12 04:00:49,208 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43977 2023-02-12 04:00:49,219 INFO [Listener at localhost.localdomain/35207] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-02-12 04:00:49,219 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-12 04:00:49,219 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-12 04:00:49,219 INFO [Listener at localhost.localdomain/35207] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-12 04:00:49,219 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-12 04:00:49,219 INFO [Listener at localhost.localdomain/35207] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-12 04:00:49,219 INFO [Listener at localhost.localdomain/35207] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-12 04:00:49,223 INFO [Listener at localhost.localdomain/35207] ipc.NettyRpcServer(120): Bind to /148.251.75.209:37065 2023-02-12 04:00:49,224 INFO [Listener at localhost.localdomain/35207] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-12 04:00:49,225 DEBUG [Listener at localhost.localdomain/35207] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-12 04:00:49,226 INFO [Listener at localhost.localdomain/35207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 04:00:49,228 INFO [Listener at localhost.localdomain/35207] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 04:00:49,229 INFO [Listener at localhost.localdomain/35207] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37065 connecting to ZooKeeper ensemble=127.0.0.1:56679 2023-02-12 04:00:49,234 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:370650x0, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-12 04:00:49,234 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ZKUtil(164): regionserver:370650x0, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-12 04:00:49,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:37065-0x10154d814950003 connected 2023-02-12 04:00:49,235 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-12 04:00:49,236 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-12 04:00:49,238 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37065 2023-02-12 04:00:49,239 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37065 2023-02-12 04:00:49,240 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37065 2023-02-12 04:00:49,242 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37065 2023-02-12 04:00:49,242 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37065 2023-02-12 04:00:49,245 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase20.apache.org,41569,1676174448151 2023-02-12 04:00:49,253 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-12 04:00:49,255 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase20.apache.org,41569,1676174448151 2023-02-12 04:00:49,272 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-12 04:00:49,272 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-12 04:00:49,272 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-12 04:00:49,272 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-12 04:00:49,273 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 04:00:49,273 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-12 04:00:49,275 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase20.apache.org,41569,1676174448151 from backup master directory 2023-02-12 04:00:49,275 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-12 04:00:49,278 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase20.apache.org,41569,1676174448151 2023-02-12 04:00:49,279 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-12 04:00:49,279 WARN [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-12 04:00:49,279 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase20.apache.org,41569,1676174448151 2023-02-12 04:00:49,282 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-12 04:00:49,283 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-12 04:00:49,378 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/hbase.id with ID: 5d052043-5bb4-4544-8b58-e193be3cb2b5 2023-02-12 04:00:49,424 INFO [master/jenkins-hbase20:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 04:00:49,440 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 04:00:49,493 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x42e84285 to 127.0.0.1:56679 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 04:00:49,525 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65966508, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 04:00:49,547 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-12 04:00:49,548 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-12 04:00:49,564 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-12 04:00:49,564 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-12 04:00:49,566 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-12 04:00:49,571 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-12 04:00:49,572 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-12 04:00:49,602 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/data/master/store-tmp 2023-02-12 04:00:49,641 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:00:49,642 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-12 04:00:49,642 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 04:00:49,642 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 04:00:49,642 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-12 04:00:49,642 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 04:00:49,642 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 04:00:49,642 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676174449641Disabling compacts and flushes for region at 1676174449642 (+1 ms)Disabling writes for close at 1676174449642Writing region close event to WAL at 1676174449642Closed at 1676174449642 2023-02-12 04:00:49,644 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/WALs/jenkins-hbase20.apache.org,41569,1676174448151 2023-02-12 04:00:49,664 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C41569%2C1676174448151, suffix=, logDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/WALs/jenkins-hbase20.apache.org,41569,1676174448151, archiveDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/oldWALs, maxLogs=10 2023-02-12 04:00:49,719 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46703,DS-1e6f0745-09f2-486f-9b8d-507ec526706c,DISK] 2023-02-12 04:00:49,719 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41527,DS-4e98e139-83ef-42ee-bcd5-93432335188f,DISK] 2023-02-12 04:00:49,719 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33265,DS-db7ce3c1-4d75-4e76-bb51-66241fa898fb,DISK] 2023-02-12 04:00:49,726 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-12 04:00:49,805 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/WALs/jenkins-hbase20.apache.org,41569,1676174448151/jenkins-hbase20.apache.org%2C41569%2C1676174448151.1676174449673 2023-02-12 04:00:49,806 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46703,DS-1e6f0745-09f2-486f-9b8d-507ec526706c,DISK], DatanodeInfoWithStorage[127.0.0.1:41527,DS-4e98e139-83ef-42ee-bcd5-93432335188f,DISK], DatanodeInfoWithStorage[127.0.0.1:33265,DS-db7ce3c1-4d75-4e76-bb51-66241fa898fb,DISK]] 2023-02-12 04:00:49,806 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:00:49,807 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:00:49,810 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-12 04:00:49,811 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-12 04:00:49,864 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-12 04:00:49,871 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-12 04:00:49,893 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-12 04:00:49,904 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:49,909 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-12 04:00:49,911 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-12 04:00:49,924 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-12 04:00:49,928 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 04:00:49,930 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64891094, jitterRate=-0.03304734826087952}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 04:00:49,930 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676174449834Initializing all the Stores at 1676174449836 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174449837 (+1 ms)Cleaning up temporary data from old regions at 1676174449916 (+79 ms)Cleaning up detritus from prior splits at 1676174449917 (+1 ms)Region opened successfully at 1676174449930 (+13 ms) 2023-02-12 04:00:49,931 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-12 04:00:49,949 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:00:49,949 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:00:49,951 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:00:49,953 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-12 04:00:49,980 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 27 msec 2023-02-12 04:00:49,981 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-12 04:00:50,002 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-12 04:00:50,008 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-12 04:00:50,029 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-12 04:00:50,032 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-12 04:00:50,033 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-12 04:00:50,037 INFO [master/jenkins-hbase20:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-12 04:00:50,041 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-12 04:00:50,044 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 04:00:50,046 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-12 04:00:50,046 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-12 04:00:50,058 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-12 04:00:50,062 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-12 04:00:50,062 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-12 04:00:50,062 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-12 04:00:50,062 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-12 04:00:50,062 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 04:00:50,063 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase20.apache.org,41569,1676174448151, sessionid=0x10154d814950000, setting cluster-up flag (Was=false) 2023-02-12 04:00:50,076 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 04:00:50,080 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-12 04:00:50,081 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,41569,1676174448151 2023-02-12 04:00:50,086 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 04:00:50,089 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-12 04:00:50,090 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,41569,1676174448151 2023-02-12 04:00:50,093 WARN [master/jenkins-hbase20:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.hbase-snapshot/.tmp 2023-02-12 04:00:50,147 INFO [RS:2;jenkins-hbase20:37065] regionserver.HRegionServer(952): ClusterId : 5d052043-5bb4-4544-8b58-e193be3cb2b5 2023-02-12 04:00:50,147 INFO [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(952): ClusterId : 5d052043-5bb4-4544-8b58-e193be3cb2b5 2023-02-12 04:00:50,147 INFO [RS:0;jenkins-hbase20:40151] regionserver.HRegionServer(952): ClusterId : 5d052043-5bb4-4544-8b58-e193be3cb2b5 2023-02-12 04:00:50,154 DEBUG [RS:0;jenkins-hbase20:40151] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-12 04:00:50,154 DEBUG [RS:1;jenkins-hbase20:43977] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-12 04:00:50,154 DEBUG [RS:2;jenkins-hbase20:37065] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-12 04:00:50,159 DEBUG [RS:2;jenkins-hbase20:37065] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-12 04:00:50,159 DEBUG [RS:1;jenkins-hbase20:43977] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-12 04:00:50,159 DEBUG [RS:2;jenkins-hbase20:37065] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-12 04:00:50,159 DEBUG [RS:0;jenkins-hbase20:40151] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-12 04:00:50,159 DEBUG [RS:1;jenkins-hbase20:43977] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-12 04:00:50,160 DEBUG [RS:0;jenkins-hbase20:40151] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-12 04:00:50,162 DEBUG [RS:2;jenkins-hbase20:37065] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-12 04:00:50,162 DEBUG [RS:0;jenkins-hbase20:40151] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-12 04:00:50,162 DEBUG [RS:1;jenkins-hbase20:43977] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-12 04:00:50,170 DEBUG [RS:0;jenkins-hbase20:40151] zookeeper.ReadOnlyZKClient(139): Connect 0x4df21505 to 127.0.0.1:56679 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 04:00:50,171 DEBUG [RS:1;jenkins-hbase20:43977] zookeeper.ReadOnlyZKClient(139): Connect 0x6a8349c7 to 127.0.0.1:56679 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 04:00:50,170 DEBUG [RS:2;jenkins-hbase20:37065] zookeeper.ReadOnlyZKClient(139): Connect 0x4c9b813c to 127.0.0.1:56679 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 04:00:50,203 DEBUG [RS:2;jenkins-hbase20:37065] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@425c66df, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 04:00:50,203 DEBUG [RS:0;jenkins-hbase20:40151] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4335f2cb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 04:00:50,204 DEBUG [RS:2;jenkins-hbase20:37065] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4663ea4c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-12 04:00:50,204 DEBUG [RS:0;jenkins-hbase20:40151] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@470a7e75, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-12 04:00:50,207 DEBUG [RS:1;jenkins-hbase20:43977] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6444aae7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 04:00:50,207 DEBUG [RS:1;jenkins-hbase20:43977] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@607b22b7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-12 04:00:50,230 DEBUG [RS:0;jenkins-hbase20:40151] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase20:40151 2023-02-12 04:00:50,232 DEBUG [RS:1;jenkins-hbase20:43977] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase20:43977 2023-02-12 04:00:50,233 DEBUG [RS:2;jenkins-hbase20:37065] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase20:37065 2023-02-12 04:00:50,235 INFO [RS:1;jenkins-hbase20:43977] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-12 04:00:50,235 INFO [RS:0;jenkins-hbase20:40151] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-12 04:00:50,236 INFO [RS:0;jenkins-hbase20:40151] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-12 04:00:50,235 INFO [RS:2;jenkins-hbase20:37065] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-12 04:00:50,236 DEBUG [RS:0;jenkins-hbase20:40151] regionserver.HRegionServer(1023): About to register with Master. 2023-02-12 04:00:50,236 INFO [RS:2;jenkins-hbase20:37065] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-12 04:00:50,236 INFO [RS:1;jenkins-hbase20:43977] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-12 04:00:50,236 DEBUG [RS:2;jenkins-hbase20:37065] regionserver.HRegionServer(1023): About to register with Master. 2023-02-12 04:00:50,236 DEBUG [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1023): About to register with Master. 2023-02-12 04:00:50,238 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-12 04:00:50,238 INFO [RS:2;jenkins-hbase20:37065] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,41569,1676174448151 with isa=jenkins-hbase20.apache.org/148.251.75.209:37065, startcode=1676174449218 2023-02-12 04:00:50,238 INFO [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,41569,1676174448151 with isa=jenkins-hbase20.apache.org/148.251.75.209:43977, startcode=1676174449184 2023-02-12 04:00:50,238 INFO [RS:0;jenkins-hbase20:40151] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,41569,1676174448151 with isa=jenkins-hbase20.apache.org/148.251.75.209:40151, startcode=1676174449139 2023-02-12 04:00:50,249 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-12 04:00:50,249 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-12 04:00:50,249 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-12 04:00:50,250 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-02-12 04:00:50,250 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase20:0, corePoolSize=10, maxPoolSize=10 2023-02-12 04:00:50,250 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,250 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-12 04:00:50,250 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,251 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174480251 2023-02-12 04:00:50,253 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-12 04:00:50,257 DEBUG [RS:1;jenkins-hbase20:43977] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-12 04:00:50,257 DEBUG [RS:2;jenkins-hbase20:37065] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-12 04:00:50,257 DEBUG [RS:0;jenkins-hbase20:40151] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-12 04:00:50,261 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-12 04:00:50,262 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-12 04:00:50,268 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-12 04:00:50,268 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-12 04:00:50,275 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-12 04:00:50,275 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-12 04:00:50,276 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-12 04:00:50,276 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-12 04:00:50,280 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,282 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-12 04:00:50,285 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-12 04:00:50,285 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-12 04:00:50,291 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-12 04:00:50,292 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-12 04:00:50,294 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676174450294,5,FailOnTimeoutGroup] 2023-02-12 04:00:50,294 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676174450294,5,FailOnTimeoutGroup] 2023-02-12 04:00:50,294 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,294 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-12 04:00:50,296 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,296 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,317 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:53625, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-12 04:00:50,317 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56227, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-12 04:00:50,317 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:43579, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-12 04:00:50,340 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-12 04:00:50,341 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-12 04:00:50,341 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2 2023-02-12 04:00:50,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41569] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:50,343 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41569] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:50,344 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:50,370 DEBUG [RS:0;jenkins-hbase20:40151] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2 2023-02-12 04:00:50,370 DEBUG [RS:0;jenkins-hbase20:40151] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:37867 2023-02-12 04:00:50,370 DEBUG [RS:0;jenkins-hbase20:40151] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-12 04:00:50,371 DEBUG [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2 2023-02-12 04:00:50,371 DEBUG [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:37867 2023-02-12 04:00:50,371 DEBUG [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-12 04:00:50,371 DEBUG [RS:2;jenkins-hbase20:37065] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2 2023-02-12 04:00:50,371 DEBUG [RS:2;jenkins-hbase20:37065] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:37867 2023-02-12 04:00:50,371 DEBUG [RS:2;jenkins-hbase20:37065] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-12 04:00:50,373 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:00:50,376 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-12 04:00:50,376 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-12 04:00:50,378 DEBUG [RS:2;jenkins-hbase20:37065] zookeeper.ZKUtil(162): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:50,378 WARN [RS:2;jenkins-hbase20:37065] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-12 04:00:50,378 DEBUG [RS:1;jenkins-hbase20:43977] zookeeper.ZKUtil(162): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:50,378 DEBUG [RS:0;jenkins-hbase20:40151] zookeeper.ZKUtil(162): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:50,378 INFO [RS:2;jenkins-hbase20:37065] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-12 04:00:50,379 WARN [RS:0;jenkins-hbase20:40151] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-12 04:00:50,378 WARN [RS:1;jenkins-hbase20:43977] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-12 04:00:50,379 INFO [RS:0;jenkins-hbase20:40151] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-12 04:00:50,379 DEBUG [RS:2;jenkins-hbase20:37065] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/WALs/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:50,379 INFO [RS:1;jenkins-hbase20:43977] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-12 04:00:50,380 DEBUG [RS:0;jenkins-hbase20:40151] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/WALs/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:50,379 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,43977,1676174449184] 2023-02-12 04:00:50,380 DEBUG [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/WALs/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:50,380 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,37065,1676174449218] 2023-02-12 04:00:50,380 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,40151,1676174449139] 2023-02-12 04:00:50,385 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/info 2023-02-12 04:00:50,387 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-12 04:00:50,389 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:50,390 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-12 04:00:50,394 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/rep_barrier 2023-02-12 04:00:50,395 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-12 04:00:50,395 DEBUG [RS:1;jenkins-hbase20:43977] zookeeper.ZKUtil(162): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:50,395 DEBUG [RS:0;jenkins-hbase20:40151] zookeeper.ZKUtil(162): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:50,396 DEBUG [RS:2;jenkins-hbase20:37065] zookeeper.ZKUtil(162): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:50,396 DEBUG [RS:1;jenkins-hbase20:43977] zookeeper.ZKUtil(162): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:50,396 DEBUG [RS:0;jenkins-hbase20:40151] zookeeper.ZKUtil(162): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:50,396 DEBUG [RS:2;jenkins-hbase20:37065] zookeeper.ZKUtil(162): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:50,396 DEBUG [RS:1;jenkins-hbase20:43977] zookeeper.ZKUtil(162): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:50,396 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:50,397 DEBUG [RS:0;jenkins-hbase20:40151] zookeeper.ZKUtil(162): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:50,397 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-12 04:00:50,397 DEBUG [RS:2;jenkins-hbase20:37065] zookeeper.ZKUtil(162): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:50,399 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/table 2023-02-12 04:00:50,400 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-12 04:00:50,401 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:50,407 DEBUG [RS:0;jenkins-hbase20:40151] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-12 04:00:50,408 DEBUG [RS:2;jenkins-hbase20:37065] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-12 04:00:50,408 DEBUG [RS:1;jenkins-hbase20:43977] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-12 04:00:50,409 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740 2023-02-12 04:00:50,409 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740 2023-02-12 04:00:50,415 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-12 04:00:50,418 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-12 04:00:50,419 INFO [RS:2;jenkins-hbase20:37065] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-12 04:00:50,419 INFO [RS:1;jenkins-hbase20:43977] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-12 04:00:50,420 INFO [RS:0;jenkins-hbase20:40151] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-12 04:00:50,427 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 04:00:50,429 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75394185, jitterRate=0.12346090376377106}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-12 04:00:50,429 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676174450374Initializing all the Stores at 1676174450376 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676174450376Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174450376Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676174450376Cleaning up temporary data from old regions at 1676174450412 (+36 ms)Cleaning up detritus from prior splits at 1676174450412Region opened successfully at 1676174450429 (+17 ms) 2023-02-12 04:00:50,429 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-12 04:00:50,429 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-12 04:00:50,429 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-12 04:00:50,429 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-12 04:00:50,429 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-12 04:00:50,431 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-12 04:00:50,431 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676174450429Disabling compacts and flushes for region at 1676174450429Disabling writes for close at 1676174450429Writing region close event to WAL at 1676174450431 (+2 ms)Closed at 1676174450431 2023-02-12 04:00:50,440 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-12 04:00:50,440 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-12 04:00:50,448 INFO [RS:2;jenkins-hbase20:37065] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-12 04:00:50,448 INFO [RS:1;jenkins-hbase20:43977] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-12 04:00:50,448 INFO [RS:0;jenkins-hbase20:40151] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-12 04:00:50,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-12 04:00:50,453 INFO [RS:1;jenkins-hbase20:43977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 04:00:50,453 INFO [RS:0;jenkins-hbase20:40151] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 04:00:50,453 INFO [RS:1;jenkins-hbase20:43977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,453 INFO [RS:2;jenkins-hbase20:37065] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 04:00:50,453 INFO [RS:0;jenkins-hbase20:40151] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,454 INFO [RS:2;jenkins-hbase20:37065] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,454 INFO [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-12 04:00:50,455 INFO [RS:0;jenkins-hbase20:40151] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-12 04:00:50,459 INFO [RS:2;jenkins-hbase20:37065] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-12 04:00:50,466 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-12 04:00:50,466 INFO [RS:2;jenkins-hbase20:37065] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,466 INFO [RS:1;jenkins-hbase20:43977] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,466 INFO [RS:0;jenkins-hbase20:40151] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,466 DEBUG [RS:2;jenkins-hbase20:37065] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,466 DEBUG [RS:1;jenkins-hbase20:43977] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:0;jenkins-hbase20:40151] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:1;jenkins-hbase20:43977] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:0;jenkins-hbase20:40151] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:1;jenkins-hbase20:43977] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:0;jenkins-hbase20:40151] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:1;jenkins-hbase20:43977] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:0;jenkins-hbase20:40151] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:1;jenkins-hbase20:43977] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:0;jenkins-hbase20:40151] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:1;jenkins-hbase20:43977] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-12 04:00:50,467 DEBUG [RS:2;jenkins-hbase20:37065] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:1;jenkins-hbase20:43977] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:2;jenkins-hbase20:37065] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:1;jenkins-hbase20:43977] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:2;jenkins-hbase20:37065] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,468 DEBUG [RS:1;jenkins-hbase20:43977] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,468 DEBUG [RS:2;jenkins-hbase20:37065] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,468 DEBUG [RS:2;jenkins-hbase20:37065] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-12 04:00:50,468 DEBUG [RS:2;jenkins-hbase20:37065] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,468 DEBUG [RS:2;jenkins-hbase20:37065] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,468 DEBUG [RS:2;jenkins-hbase20:37065] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,468 DEBUG [RS:2;jenkins-hbase20:37065] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,467 DEBUG [RS:0;jenkins-hbase20:40151] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-02-12 04:00:50,468 DEBUG [RS:1;jenkins-hbase20:43977] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,469 DEBUG [RS:0;jenkins-hbase20:40151] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,469 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-12 04:00:50,469 DEBUG [RS:0;jenkins-hbase20:40151] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,469 DEBUG [RS:0;jenkins-hbase20:40151] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,469 DEBUG [RS:0;jenkins-hbase20:40151] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-02-12 04:00:50,471 INFO [RS:2;jenkins-hbase20:37065] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,471 INFO [RS:1;jenkins-hbase20:43977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,475 INFO [RS:2;jenkins-hbase20:37065] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,475 INFO [RS:1;jenkins-hbase20:43977] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,475 INFO [RS:2;jenkins-hbase20:37065] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,475 INFO [RS:1;jenkins-hbase20:43977] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,479 INFO [RS:0;jenkins-hbase20:40151] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,479 INFO [RS:0;jenkins-hbase20:40151] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,479 INFO [RS:0;jenkins-hbase20:40151] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,493 INFO [RS:0;jenkins-hbase20:40151] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-12 04:00:50,493 INFO [RS:2;jenkins-hbase20:37065] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-12 04:00:50,493 INFO [RS:1;jenkins-hbase20:43977] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-12 04:00:50,496 INFO [RS:0;jenkins-hbase20:40151] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,40151,1676174449139-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,496 INFO [RS:1;jenkins-hbase20:43977] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,43977,1676174449184-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,496 INFO [RS:2;jenkins-hbase20:37065] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,37065,1676174449218-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:50,516 INFO [RS:1;jenkins-hbase20:43977] regionserver.Replication(203): jenkins-hbase20.apache.org,43977,1676174449184 started 2023-02-12 04:00:50,516 INFO [RS:0;jenkins-hbase20:40151] regionserver.Replication(203): jenkins-hbase20.apache.org,40151,1676174449139 started 2023-02-12 04:00:50,516 INFO [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,43977,1676174449184, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:43977, sessionid=0x10154d814950002 2023-02-12 04:00:50,516 INFO [RS:0;jenkins-hbase20:40151] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,40151,1676174449139, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:40151, sessionid=0x10154d814950001 2023-02-12 04:00:50,516 DEBUG [RS:0;jenkins-hbase20:40151] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-12 04:00:50,516 DEBUG [RS:1;jenkins-hbase20:43977] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-12 04:00:50,516 DEBUG [RS:0;jenkins-hbase20:40151] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:50,516 DEBUG [RS:1;jenkins-hbase20:43977] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:50,517 DEBUG [RS:0;jenkins-hbase20:40151] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,40151,1676174449139' 2023-02-12 04:00:50,517 DEBUG [RS:1;jenkins-hbase20:43977] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,43977,1676174449184' 2023-02-12 04:00:50,518 DEBUG [RS:1;jenkins-hbase20:43977] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:50,517 DEBUG [RS:0;jenkins-hbase20:40151] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:50,518 DEBUG [RS:1;jenkins-hbase20:43977] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:50,518 DEBUG [RS:0;jenkins-hbase20:40151] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:50,519 DEBUG [RS:1;jenkins-hbase20:43977] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-12 04:00:50,519 DEBUG [RS:1;jenkins-hbase20:43977] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-12 04:00:50,519 DEBUG [RS:0;jenkins-hbase20:40151] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-12 04:00:50,519 DEBUG [RS:1;jenkins-hbase20:43977] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:50,519 DEBUG [RS:0;jenkins-hbase20:40151] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-12 04:00:50,519 DEBUG [RS:1;jenkins-hbase20:43977] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,43977,1676174449184' 2023-02-12 04:00:50,520 DEBUG [RS:1;jenkins-hbase20:43977] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-12 04:00:50,519 DEBUG [RS:0;jenkins-hbase20:40151] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:50,520 DEBUG [RS:0;jenkins-hbase20:40151] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,40151,1676174449139' 2023-02-12 04:00:50,520 DEBUG [RS:0;jenkins-hbase20:40151] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-12 04:00:50,520 DEBUG [RS:1;jenkins-hbase20:43977] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-12 04:00:50,520 DEBUG [RS:0;jenkins-hbase20:40151] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-12 04:00:50,520 INFO [RS:2;jenkins-hbase20:37065] regionserver.Replication(203): jenkins-hbase20.apache.org,37065,1676174449218 started 2023-02-12 04:00:50,520 DEBUG [RS:1;jenkins-hbase20:43977] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-12 04:00:50,520 INFO [RS:2;jenkins-hbase20:37065] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,37065,1676174449218, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:37065, sessionid=0x10154d814950003 2023-02-12 04:00:50,520 INFO [RS:1;jenkins-hbase20:43977] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-12 04:00:50,521 DEBUG [RS:2;jenkins-hbase20:37065] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-12 04:00:50,520 DEBUG [RS:0;jenkins-hbase20:40151] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-12 04:00:50,521 DEBUG [RS:2;jenkins-hbase20:37065] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:50,521 INFO [RS:1;jenkins-hbase20:43977] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-12 04:00:50,521 DEBUG [RS:2;jenkins-hbase20:37065] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,37065,1676174449218' 2023-02-12 04:00:50,521 INFO [RS:0;jenkins-hbase20:40151] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-12 04:00:50,521 DEBUG [RS:2;jenkins-hbase20:37065] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:50,522 INFO [RS:0;jenkins-hbase20:40151] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-12 04:00:50,523 DEBUG [RS:2;jenkins-hbase20:37065] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:50,523 DEBUG [RS:2;jenkins-hbase20:37065] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-12 04:00:50,524 DEBUG [RS:2;jenkins-hbase20:37065] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-12 04:00:50,524 DEBUG [RS:2;jenkins-hbase20:37065] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:50,524 DEBUG [RS:2;jenkins-hbase20:37065] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,37065,1676174449218' 2023-02-12 04:00:50,524 DEBUG [RS:2;jenkins-hbase20:37065] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-12 04:00:50,524 DEBUG [RS:2;jenkins-hbase20:37065] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-12 04:00:50,525 DEBUG [RS:2;jenkins-hbase20:37065] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-12 04:00:50,525 INFO [RS:2;jenkins-hbase20:37065] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-12 04:00:50,525 INFO [RS:2;jenkins-hbase20:37065] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-12 04:00:50,622 DEBUG [jenkins-hbase20:41569] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-12 04:00:50,626 DEBUG [jenkins-hbase20:41569] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-12 04:00:50,630 INFO [RS:2;jenkins-hbase20:37065] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C37065%2C1676174449218, suffix=, logDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/WALs/jenkins-hbase20.apache.org,37065,1676174449218, archiveDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/oldWALs, maxLogs=32 2023-02-12 04:00:50,632 INFO [RS:0;jenkins-hbase20:40151] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C40151%2C1676174449139, suffix=, logDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/WALs/jenkins-hbase20.apache.org,40151,1676174449139, archiveDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/oldWALs, maxLogs=32 2023-02-12 04:00:50,630 INFO [RS:1;jenkins-hbase20:43977] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C43977%2C1676174449184, suffix=, logDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/WALs/jenkins-hbase20.apache.org,43977,1676174449184, archiveDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/oldWALs, maxLogs=32 2023-02-12 04:00:50,634 DEBUG [jenkins-hbase20:41569] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-12 04:00:50,634 DEBUG [jenkins-hbase20:41569] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-12 04:00:50,634 DEBUG [jenkins-hbase20:41569] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-12 04:00:50,634 DEBUG [jenkins-hbase20:41569] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-12 04:00:50,636 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,43977,1676174449184, state=OPENING 2023-02-12 04:00:50,642 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-12 04:00:50,644 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 04:00:50,644 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-12 04:00:50,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:00:50,664 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41527,DS-4e98e139-83ef-42ee-bcd5-93432335188f,DISK] 2023-02-12 04:00:50,664 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33265,DS-db7ce3c1-4d75-4e76-bb51-66241fa898fb,DISK] 2023-02-12 04:00:50,665 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46703,DS-1e6f0745-09f2-486f-9b8d-507ec526706c,DISK] 2023-02-12 04:00:50,675 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41527,DS-4e98e139-83ef-42ee-bcd5-93432335188f,DISK] 2023-02-12 04:00:50,683 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41527,DS-4e98e139-83ef-42ee-bcd5-93432335188f,DISK] 2023-02-12 04:00:50,683 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33265,DS-db7ce3c1-4d75-4e76-bb51-66241fa898fb,DISK] 2023-02-12 04:00:50,683 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46703,DS-1e6f0745-09f2-486f-9b8d-507ec526706c,DISK] 2023-02-12 04:00:50,684 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46703,DS-1e6f0745-09f2-486f-9b8d-507ec526706c,DISK] 2023-02-12 04:00:50,684 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33265,DS-db7ce3c1-4d75-4e76-bb51-66241fa898fb,DISK] 2023-02-12 04:00:50,697 INFO [RS:0;jenkins-hbase20:40151] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/WALs/jenkins-hbase20.apache.org,40151,1676174449139/jenkins-hbase20.apache.org%2C40151%2C1676174449139.1676174450636 2023-02-12 04:00:50,698 DEBUG [RS:0;jenkins-hbase20:40151] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41527,DS-4e98e139-83ef-42ee-bcd5-93432335188f,DISK], DatanodeInfoWithStorage[127.0.0.1:46703,DS-1e6f0745-09f2-486f-9b8d-507ec526706c,DISK], DatanodeInfoWithStorage[127.0.0.1:33265,DS-db7ce3c1-4d75-4e76-bb51-66241fa898fb,DISK]] 2023-02-12 04:00:50,705 INFO [RS:2;jenkins-hbase20:37065] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/WALs/jenkins-hbase20.apache.org,37065,1676174449218/jenkins-hbase20.apache.org%2C37065%2C1676174449218.1676174450636 2023-02-12 04:00:50,705 INFO [RS:1;jenkins-hbase20:43977] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/WALs/jenkins-hbase20.apache.org,43977,1676174449184/jenkins-hbase20.apache.org%2C43977%2C1676174449184.1676174450636 2023-02-12 04:00:50,707 DEBUG [RS:2;jenkins-hbase20:37065] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46703,DS-1e6f0745-09f2-486f-9b8d-507ec526706c,DISK], DatanodeInfoWithStorage[127.0.0.1:33265,DS-db7ce3c1-4d75-4e76-bb51-66241fa898fb,DISK], DatanodeInfoWithStorage[127.0.0.1:41527,DS-4e98e139-83ef-42ee-bcd5-93432335188f,DISK]] 2023-02-12 04:00:50,707 DEBUG [RS:1;jenkins-hbase20:43977] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41527,DS-4e98e139-83ef-42ee-bcd5-93432335188f,DISK], DatanodeInfoWithStorage[127.0.0.1:33265,DS-db7ce3c1-4d75-4e76-bb51-66241fa898fb,DISK], DatanodeInfoWithStorage[127.0.0.1:46703,DS-1e6f0745-09f2-486f-9b8d-507ec526706c,DISK]] 2023-02-12 04:00:50,862 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:50,865 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-12 04:00:50,868 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52620, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-12 04:00:50,880 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-12 04:00:50,881 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-12 04:00:50,885 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C43977%2C1676174449184.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/WALs/jenkins-hbase20.apache.org,43977,1676174449184, archiveDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/oldWALs, maxLogs=32 2023-02-12 04:00:50,904 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41527,DS-4e98e139-83ef-42ee-bcd5-93432335188f,DISK] 2023-02-12 04:00:50,905 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33265,DS-db7ce3c1-4d75-4e76-bb51-66241fa898fb,DISK] 2023-02-12 04:00:50,906 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46703,DS-1e6f0745-09f2-486f-9b8d-507ec526706c,DISK] 2023-02-12 04:00:50,913 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/WALs/jenkins-hbase20.apache.org,43977,1676174449184/jenkins-hbase20.apache.org%2C43977%2C1676174449184.meta.1676174450887.meta 2023-02-12 04:00:50,913 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41527,DS-4e98e139-83ef-42ee-bcd5-93432335188f,DISK], DatanodeInfoWithStorage[127.0.0.1:33265,DS-db7ce3c1-4d75-4e76-bb51-66241fa898fb,DISK], DatanodeInfoWithStorage[127.0.0.1:46703,DS-1e6f0745-09f2-486f-9b8d-507ec526706c,DISK]] 2023-02-12 04:00:50,913 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:00:50,918 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:00:50,918 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-12 04:00:50,938 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-12 04:00:50,940 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-12 04:00:50,944 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-12 04:00:50,944 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:00:50,944 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-12 04:00:50,944 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-12 04:00:50,947 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-12 04:00:50,949 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/info 2023-02-12 04:00:50,949 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/info 2023-02-12 04:00:50,949 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-12 04:00:50,950 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:50,950 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-12 04:00:50,952 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/rep_barrier 2023-02-12 04:00:50,952 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/rep_barrier 2023-02-12 04:00:50,953 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-12 04:00:50,954 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:50,954 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-12 04:00:50,955 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/table 2023-02-12 04:00:50,955 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/table 2023-02-12 04:00:50,956 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-12 04:00:50,957 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:50,959 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740 2023-02-12 04:00:50,962 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740 2023-02-12 04:00:50,966 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-12 04:00:50,969 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-12 04:00:50,971 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60698785, jitterRate=-0.09551762044429779}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-12 04:00:50,971 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676174450944Writing region info on filesystem at 1676174450945 (+1 ms)Initializing all the Stores at 1676174450946 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676174450946Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174450947 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676174450947Cleaning up temporary data from old regions at 1676174450964 (+17 ms)Cleaning up detritus from prior splits at 1676174450965 (+1 ms)Running coprocessor post-open hooks at 1676174450971 (+6 ms)Region opened successfully at 1676174450971 2023-02-12 04:00:50,984 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676174450856 2023-02-12 04:00:51,004 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-12 04:00:51,005 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-12 04:00:51,005 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,43977,1676174449184, state=OPEN 2023-02-12 04:00:51,008 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-12 04:00:51,008 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-12 04:00:51,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-12 04:00:51,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,43977,1676174449184 in 360 msec 2023-02-12 04:00:51,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-12 04:00:51,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 566 msec 2023-02-12 04:00:51,026 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 891 msec 2023-02-12 04:00:51,026 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676174449269, completionTime=-1 2023-02-12 04:00:51,027 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-12 04:00:51,027 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:00:51,089 DEBUG [hconnection-0x6c2e965c-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 04:00:51,092 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52622, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 04:00:51,108 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:00:51,109 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174511109 2023-02-12 04:00:51,109 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174571109 2023-02-12 04:00:51,109 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 81 msec 2023-02-12 04:00:51,134 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,41569,1676174448151-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:51,134 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,41569,1676174448151-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:51,134 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,41569,1676174448151-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:51,136 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase20:41569, period=300000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:51,136 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-12 04:00:51,149 DEBUG [master/jenkins-hbase20:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-12 04:00:51,157 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-12 04:00:51,159 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-12 04:00:51,170 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-12 04:00:51,173 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 04:00:51,176 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 04:00:51,198 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/hbase/namespace/6b2660cb506f26cb2f51dda8314eff58 2023-02-12 04:00:51,201 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/hbase/namespace/6b2660cb506f26cb2f51dda8314eff58 empty. 2023-02-12 04:00:51,202 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/hbase/namespace/6b2660cb506f26cb2f51dda8314eff58 2023-02-12 04:00:51,202 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-12 04:00:51,241 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-12 04:00:51,243 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 6b2660cb506f26cb2f51dda8314eff58, NAME => 'hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp 2023-02-12 04:00:51,267 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:00:51,267 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 6b2660cb506f26cb2f51dda8314eff58, disabling compactions & flushes 2023-02-12 04:00:51,267 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. 2023-02-12 04:00:51,268 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. 2023-02-12 04:00:51,268 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. after waiting 0 ms 2023-02-12 04:00:51,268 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. 2023-02-12 04:00:51,268 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. 2023-02-12 04:00:51,268 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 6b2660cb506f26cb2f51dda8314eff58: Waiting for close lock at 1676174451267Disabling compacts and flushes for region at 1676174451267Disabling writes for close at 1676174451268 (+1 ms)Writing region close event to WAL at 1676174451268Closed at 1676174451268 2023-02-12 04:00:51,272 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 04:00:51,287 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676174451275"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174451275"}]},"ts":"1676174451275"} 2023-02-12 04:00:51,313 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 04:00:51,315 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 04:00:51,319 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174451315"}]},"ts":"1676174451315"} 2023-02-12 04:00:51,324 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-12 04:00:51,328 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-12 04:00:51,329 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-12 04:00:51,329 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-12 04:00:51,329 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-12 04:00:51,329 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-12 04:00:51,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6b2660cb506f26cb2f51dda8314eff58, ASSIGN}] 2023-02-12 04:00:51,335 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=6b2660cb506f26cb2f51dda8314eff58, ASSIGN 2023-02-12 04:00:51,336 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=6b2660cb506f26cb2f51dda8314eff58, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,43977,1676174449184; forceNewPlan=false, retain=false 2023-02-12 04:00:51,488 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-12 04:00:51,489 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6b2660cb506f26cb2f51dda8314eff58, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:51,490 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676174451489"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174451489"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174451489"}]},"ts":"1676174451489"} 2023-02-12 04:00:51,494 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 6b2660cb506f26cb2f51dda8314eff58, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:00:51,657 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. 2023-02-12 04:00:51,659 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6b2660cb506f26cb2f51dda8314eff58, NAME => 'hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:00:51,662 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:00:51,663 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 6b2660cb506f26cb2f51dda8314eff58 2023-02-12 04:00:51,663 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:00:51,663 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 6b2660cb506f26cb2f51dda8314eff58 2023-02-12 04:00:51,663 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 6b2660cb506f26cb2f51dda8314eff58 2023-02-12 04:00:51,665 INFO [StoreOpener-6b2660cb506f26cb2f51dda8314eff58-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 6b2660cb506f26cb2f51dda8314eff58 2023-02-12 04:00:51,667 DEBUG [StoreOpener-6b2660cb506f26cb2f51dda8314eff58-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/namespace/6b2660cb506f26cb2f51dda8314eff58/info 2023-02-12 04:00:51,667 DEBUG [StoreOpener-6b2660cb506f26cb2f51dda8314eff58-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/namespace/6b2660cb506f26cb2f51dda8314eff58/info 2023-02-12 04:00:51,668 INFO [StoreOpener-6b2660cb506f26cb2f51dda8314eff58-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b2660cb506f26cb2f51dda8314eff58 columnFamilyName info 2023-02-12 04:00:51,669 INFO [StoreOpener-6b2660cb506f26cb2f51dda8314eff58-1] regionserver.HStore(310): Store=6b2660cb506f26cb2f51dda8314eff58/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:51,671 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/namespace/6b2660cb506f26cb2f51dda8314eff58 2023-02-12 04:00:51,673 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/namespace/6b2660cb506f26cb2f51dda8314eff58 2023-02-12 04:00:51,677 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 6b2660cb506f26cb2f51dda8314eff58 2023-02-12 04:00:51,681 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/namespace/6b2660cb506f26cb2f51dda8314eff58/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 04:00:51,682 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 6b2660cb506f26cb2f51dda8314eff58; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73812523, jitterRate=0.09989230334758759}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 04:00:51,682 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 6b2660cb506f26cb2f51dda8314eff58: Running coprocessor pre-open hook at 1676174451663Writing region info on filesystem at 1676174451663Initializing all the Stores at 1676174451665 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676174451665Cleaning up temporary data from old regions at 1676174451673 (+8 ms)Cleaning up detritus from prior splits at 1676174451674 (+1 ms)Running coprocessor post-open hooks at 1676174451682 (+8 ms)Region opened successfully at 1676174451682 2023-02-12 04:00:51,686 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58., pid=6, masterSystemTime=1676174451648 2023-02-12 04:00:51,691 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. 2023-02-12 04:00:51,691 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. 2023-02-12 04:00:51,692 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:51,693 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676174451692"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174451692"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174451692"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174451692"}]},"ts":"1676174451692"} 2023-02-12 04:00:51,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-12 04:00:51,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 6b2660cb506f26cb2f51dda8314eff58, server=jenkins-hbase20.apache.org,43977,1676174449184 in 204 msec 2023-02-12 04:00:51,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-12 04:00:51,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=6b2660cb506f26cb2f51dda8314eff58, ASSIGN in 372 msec 2023-02-12 04:00:51,709 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 04:00:51,709 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174451709"}]},"ts":"1676174451709"} 2023-02-12 04:00:51,712 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-12 04:00:51,715 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 04:00:51,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 554 msec 2023-02-12 04:00:51,773 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-12 04:00:51,775 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-12 04:00:51,775 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 04:00:51,806 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-12 04:00:51,850 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-12 04:00:51,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 55 msec 2023-02-12 04:00:51,865 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:00:51,878 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-12 04:00:51,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 18 msec 2023-02-12 04:00:51,898 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-12 04:00:51,899 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-12 04:00:51,900 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.620sec 2023-02-12 04:00:51,903 INFO [master/jenkins-hbase20:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-12 04:00:51,904 INFO [master/jenkins-hbase20:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-12 04:00:51,904 INFO [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-12 04:00:51,906 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,41569,1676174448151-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-12 04:00:51,908 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,41569,1676174448151-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-12 04:00:51,916 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-12 04:00:51,952 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ReadOnlyZKClient(139): Connect 0x11de39c7 to 127.0.0.1:56679 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 04:00:51,957 DEBUG [Listener at localhost.localdomain/35207] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@167d9f41, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 04:00:51,970 DEBUG [hconnection-0x19f60b37-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 04:00:51,983 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52638, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 04:00:51,991 INFO [Listener at localhost.localdomain/35207] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase20.apache.org,41569,1676174448151 2023-02-12 04:00:52,018 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=358, OpenFileDescriptor=691, MaxFileDescriptor=60000, SystemLoadAverage=213, ProcessCount=165, AvailableMemoryMB=4326 2023-02-12 04:00:52,021 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:00:52,030 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 04:00:52,033 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:55886, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 04:00:52,044 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-12 04:00:52,045 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 04:00:52,046 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-12 04:00:52,051 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,054 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 04:00:52,056 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 04:00:52,059 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a 2023-02-12 04:00:52,060 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a empty. 2023-02-12 04:00:52,062 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a 2023-02-12 04:00:52,062 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-12 04:00:52,088 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-12 04:00:52,089 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4157975c3727721623be5c93597b743a, NAME => 'testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp 2023-02-12 04:00:52,111 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:00:52,112 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing 4157975c3727721623be5c93597b743a, disabling compactions & flushes 2023-02-12 04:00:52,112 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:52,112 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:52,112 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. after waiting 0 ms 2023-02-12 04:00:52,112 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:52,112 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:52,112 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 4157975c3727721623be5c93597b743a: Waiting for close lock at 1676174452112Disabling compacts and flushes for region at 1676174452112Disabling writes for close at 1676174452112Writing region close event to WAL at 1676174452112Closed at 1676174452112 2023-02-12 04:00:52,116 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 04:00:52,118 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676174452118"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174452118"}]},"ts":"1676174452118"} 2023-02-12 04:00:52,121 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 04:00:52,122 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 04:00:52,123 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174452122"}]},"ts":"1676174452122"} 2023-02-12 04:00:52,125 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-12 04:00:52,129 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-12 04:00:52,130 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-12 04:00:52,130 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-12 04:00:52,130 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-12 04:00:52,130 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-12 04:00:52,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4157975c3727721623be5c93597b743a, ASSIGN}] 2023-02-12 04:00:52,134 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4157975c3727721623be5c93597b743a, ASSIGN 2023-02-12 04:00:52,135 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4157975c3727721623be5c93597b743a, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,43977,1676174449184; forceNewPlan=false, retain=false 2023-02-12 04:00:52,286 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-12 04:00:52,287 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=4157975c3727721623be5c93597b743a, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,287 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676174452287"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174452287"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174452287"}]},"ts":"1676174452287"} 2023-02-12 04:00:52,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 4157975c3727721623be5c93597b743a, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:00:52,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:52,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4157975c3727721623be5c93597b743a, NAME => 'testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:00:52,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:00:52,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 4157975c3727721623be5c93597b743a 2023-02-12 04:00:52,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:00:52,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 4157975c3727721623be5c93597b743a 2023-02-12 04:00:52,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 4157975c3727721623be5c93597b743a 2023-02-12 04:00:52,453 INFO [StoreOpener-4157975c3727721623be5c93597b743a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4157975c3727721623be5c93597b743a 2023-02-12 04:00:52,455 DEBUG [StoreOpener-4157975c3727721623be5c93597b743a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1 2023-02-12 04:00:52,455 DEBUG [StoreOpener-4157975c3727721623be5c93597b743a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1 2023-02-12 04:00:52,456 INFO [StoreOpener-4157975c3727721623be5c93597b743a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4157975c3727721623be5c93597b743a columnFamilyName cf1 2023-02-12 04:00:52,457 INFO [StoreOpener-4157975c3727721623be5c93597b743a-1] regionserver.HStore(310): Store=4157975c3727721623be5c93597b743a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:52,457 INFO [StoreOpener-4157975c3727721623be5c93597b743a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4157975c3727721623be5c93597b743a 2023-02-12 04:00:52,459 DEBUG [StoreOpener-4157975c3727721623be5c93597b743a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2 2023-02-12 04:00:52,459 DEBUG [StoreOpener-4157975c3727721623be5c93597b743a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2 2023-02-12 04:00:52,460 INFO [StoreOpener-4157975c3727721623be5c93597b743a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4157975c3727721623be5c93597b743a columnFamilyName cf2 2023-02-12 04:00:52,460 INFO [StoreOpener-4157975c3727721623be5c93597b743a-1] regionserver.HStore(310): Store=4157975c3727721623be5c93597b743a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:52,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a 2023-02-12 04:00:52,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a 2023-02-12 04:00:52,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:00:52,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 4157975c3727721623be5c93597b743a 2023-02-12 04:00:52,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 04:00:52,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 4157975c3727721623be5c93597b743a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65762417, jitterRate=-0.020063623785972595}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:00:52,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 4157975c3727721623be5c93597b743a: Running coprocessor pre-open hook at 1676174452451Writing region info on filesystem at 1676174452451Initializing all the Stores at 1676174452452 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174452452Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174452453 (+1 ms)Cleaning up temporary data from old regions at 1676174452465 (+12 ms)Cleaning up detritus from prior splits at 1676174452466 (+1 ms)Running coprocessor post-open hooks at 1676174452473 (+7 ms)Region opened successfully at 1676174452473 2023-02-12 04:00:52,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a., pid=11, masterSystemTime=1676174452443 2023-02-12 04:00:52,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:52,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:52,478 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=4157975c3727721623be5c93597b743a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,480 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676174452478"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174452478"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174452478"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174452478"}]},"ts":"1676174452478"} 2023-02-12 04:00:52,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-12 04:00:52,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 4157975c3727721623be5c93597b743a, server=jenkins-hbase20.apache.org,43977,1676174449184 in 193 msec 2023-02-12 04:00:52,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-12 04:00:52,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4157975c3727721623be5c93597b743a, ASSIGN in 356 msec 2023-02-12 04:00:52,492 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 04:00:52,492 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174452492"}]},"ts":"1676174452492"} 2023-02-12 04:00:52,495 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-12 04:00:52,498 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 04:00:52,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 449 msec 2023-02-12 04:00:52,577 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:00:52,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,584 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-12 04:00:52,584 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:52,584 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-12 04:00:52,584 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-12 04:00:52,585 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,585 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,586 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:52,586 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,586 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:52,586 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:52,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:52,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:52,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:52,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:52,587 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:52,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:52,587 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:00:52,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,588 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:52,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:52,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:52,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,589 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,589 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,589 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,590 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:52,590 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:52,590 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:52,590 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:52,590 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:52,590 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:52,590 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:52,591 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:52,591 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:52,591 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:52,591 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:52,591 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:52,591 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:52,593 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:00:52,593 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:52,593 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:00:52,593 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. started... 2023-02-12 04:00:52,594 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4157975c3727721623be5c93597b743a 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-12 04:00:52,598 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,598 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,598 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,598 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,598 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:52,598 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:52,660 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/6de65c4744c743f794c4fad94c19d8af 2023-02-12 04:00:52,727 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/97c4990de03f488a9deb0213c7a09b8a 2023-02-12 04:00:52,738 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/6de65c4744c743f794c4fad94c19d8af as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/6de65c4744c743f794c4fad94c19d8af 2023-02-12 04:00:52,750 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/6de65c4744c743f794c4fad94c19d8af, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:00:52,753 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/97c4990de03f488a9deb0213c7a09b8a as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/97c4990de03f488a9deb0213c7a09b8a 2023-02-12 04:00:52,763 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/97c4990de03f488a9deb0213c7a09b8a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:00:52,765 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 4157975c3727721623be5c93597b743a in 171ms, sequenceid=6, compaction requested=false 2023-02-12 04:00:52,766 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-12 04:00:52,768 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4157975c3727721623be5c93597b743a: 2023-02-12 04:00:52,768 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:52,768 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:00:52,769 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:00:52,769 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,769 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:52,769 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:52,771 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,771 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:52,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:52,771 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,771 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:52,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:52,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:52,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:52,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:52,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:52,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-12 04:00:52,774 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-12 04:00:52,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65baf88f[Count = 0] remaining members to acquire global barrier 2023-02-12 04:00:52,774 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,782 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,782 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,782 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,782 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:52,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,782 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,783 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:00:52,782 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:52,783 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:52,783 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:52,782 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:52,783 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:00:52,783 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:52,783 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:00:52,783 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:00:52,787 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:52,787 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:52,787 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:52,787 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:52,787 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,787 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:52,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,787 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:52,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:52,788 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:52,787 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:52,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:52,788 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:52,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:52,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:52,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:52,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:52,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:52,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:52,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:00:52,793 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-12 04:00:52,793 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:00:52,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:00:52,793 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:00:52,793 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,793 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:00:52,797 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,797 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:52,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:52,797 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,797 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,798 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:52,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,797 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:52,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,798 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:52,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,798 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:52,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:52,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:52,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:52,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:52,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:52,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:52,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,799 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:52,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:52,799 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:52,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:52,801 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,801 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:52,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,802 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:52,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:52,828 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:52,828 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:52,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:52,828 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,828 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:52,828 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:52,828 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-12 04:00:52,829 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:00:52,828 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:52,828 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:52,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:52,829 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:52,828 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,828 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:52,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:52,829 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,830 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:52,830 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,830 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:52,830 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,830 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,830 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:52,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:52,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:52,829 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:52,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:52,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:52,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:52,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:52,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:52,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:52,831 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:00:52,832 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:00:52,932 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:00:52,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:00:52,968 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:00:52,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:00:52,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,972 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-12 04:00:52,972 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:52,973 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-12 04:00:52,973 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-12 04:00:52,973 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,973 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,974 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:52,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:52,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:52,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:52,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:52,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:52,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:52,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:52,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:52,975 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:52,975 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:00:52,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:52,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,976 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:52,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:52,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,977 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:52,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,977 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,977 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:52,977 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:52,977 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:52,977 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,977 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:52,978 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:52,978 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:52,978 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:52,978 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:52,978 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,978 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:52,978 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:52,979 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:52,978 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:52,979 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:52,979 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:52,978 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:52,979 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:00:52,979 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:00:52,979 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:52,979 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. started... 2023-02-12 04:00:52,980 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4157975c3727721623be5c93597b743a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:00:52,986 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,989 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,989 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,989 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:52,990 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:52,990 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:52,999 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/d6bf10f37b1c455da3101db2495f96e8 2023-02-12 04:00:53,029 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/38aef2832cd0444aaabd6f79c51fa0e3 2023-02-12 04:00:53,038 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/d6bf10f37b1c455da3101db2495f96e8 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/d6bf10f37b1c455da3101db2495f96e8 2023-02-12 04:00:53,047 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/d6bf10f37b1c455da3101db2495f96e8, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:00:53,049 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/38aef2832cd0444aaabd6f79c51fa0e3 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/38aef2832cd0444aaabd6f79c51fa0e3 2023-02-12 04:00:53,059 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/38aef2832cd0444aaabd6f79c51fa0e3, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:00:53,061 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4157975c3727721623be5c93597b743a in 81ms, sequenceid=19, compaction requested=false 2023-02-12 04:00:53,061 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4157975c3727721623be5c93597b743a: 2023-02-12 04:00:53,061 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:53,061 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:00:53,062 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:00:53,062 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,062 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:53,062 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:53,063 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,063 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:53,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:53,064 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,064 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:53,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:53,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:53,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:53,066 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-12 04:00:53,066 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5518c113[Count = 0] remaining members to acquire global barrier 2023-02-12 04:00:53,066 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-12 04:00:53,066 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,067 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,068 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,068 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:00:53,068 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,068 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:53,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:00:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:00:53,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:53,068 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:00:53,070 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:53,070 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,070 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:53,070 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:53,070 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:53,070 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,070 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,070 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:53,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:53,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:53,070 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,071 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:53,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:53,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:53,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:53,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:00:53,076 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:00:53,076 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-12 04:00:53,076 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:00:53,076 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:00:53,076 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,076 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:00:53,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:53,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:53,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,082 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:53,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,083 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,083 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:53,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,086 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,086 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,087 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:53,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,094 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,094 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,094 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,094 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,094 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,094 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,094 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,095 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,094 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,095 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,094 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-12 04:00:53,095 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:00:53,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,094 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,095 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,096 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,096 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,096 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,096 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:00:53,096 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,096 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,096 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:00:53,096 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,196 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:00:53,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:00:53,226 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:00:53,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:00:53,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,229 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-12 04:00:53,230 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,230 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-12 04:00:53,230 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-12 04:00:53,231 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,231 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,232 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,232 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,232 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,232 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,232 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,232 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:00:53,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,233 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,233 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,233 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,233 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:53,233 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,233 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,233 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:53,233 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:53,233 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,234 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,234 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:53,234 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:53,234 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,234 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:53,234 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:53,234 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,234 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:53,234 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,235 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:53,235 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:53,235 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:53,235 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:53,235 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:53,235 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,236 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:53,236 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:00:53,236 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:00:53,236 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:53,236 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. started... 2023-02-12 04:00:53,237 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4157975c3727721623be5c93597b743a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:00:53,253 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,255 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3 2023-02-12 04:00:53,256 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,256 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,256 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:53,256 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,256 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:53,280 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/b2a62742081b439b85448ce84abdf9d2 2023-02-12 04:00:53,289 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3 2023-02-12 04:00:53,297 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:00:53,298 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/b2a62742081b439b85448ce84abdf9d2 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/b2a62742081b439b85448ce84abdf9d2 2023-02-12 04:00:53,308 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/b2a62742081b439b85448ce84abdf9d2, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:00:53,309 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4157975c3727721623be5c93597b743a in 72ms, sequenceid=32, compaction requested=true 2023-02-12 04:00:53,309 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4157975c3727721623be5c93597b743a: 2023-02-12 04:00:53,309 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:53,309 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:00:53,310 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:00:53,310 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,310 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:53,310 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:53,311 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,311 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:53,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:53,312 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,312 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:53,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:53,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:53,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:53,314 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-12 04:00:53,314 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12434553[Count = 0] remaining members to acquire global barrier 2023-02-12 04:00:53,314 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-12 04:00:53,314 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,315 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,315 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,315 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,315 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:53,315 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:53,315 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,315 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:00:53,315 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:53,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,316 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:00:53,315 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:53,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,316 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:53,316 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:53,316 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:00:53,316 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:00:53,317 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:53,317 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,317 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:53,318 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:53,318 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,318 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:53,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:53,318 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:53,319 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:53,319 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,319 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:53,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:53,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:53,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:53,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:00:53,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:00:53,323 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-12 04:00:53,323 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:00:53,323 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:00:53,323 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,323 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:00:53,331 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,331 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,331 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,331 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,331 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:53,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:53,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,331 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,332 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,331 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:53,332 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,333 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:53,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:53,335 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,335 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,336 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,340 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,340 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,340 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,340 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,340 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,340 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,340 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,341 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,341 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,341 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,341 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,341 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,341 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,341 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,341 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,340 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-12 04:00:53,341 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:00:53,340 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,340 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,340 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,342 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:00:53,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,342 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:00:53,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,442 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:00:53,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:00:53,472 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:00:53,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:00:53,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,474 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-12 04:00:53,474 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,475 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-12 04:00:53,475 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-12 04:00:53,476 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,476 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,477 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,477 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,477 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,477 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,477 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,477 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:00:53,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:53,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:53,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:53,479 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,479 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,479 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,479 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:53,479 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:53,479 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:53,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,479 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:53,479 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:53,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,479 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,479 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,480 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:53,480 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:53,479 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:53,480 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:53,480 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:53,480 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,482 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,482 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,482 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:53,489 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:00:53,490 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:53,490 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:53,490 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:53,490 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:53,490 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. started... 2023-02-12 04:00:53,490 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:00:53,491 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4157975c3727721623be5c93597b743a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:00:53,498 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,498 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,498 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:53,521 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/775dbbdc63af49379602accbfda8c0a8 2023-02-12 04:00:53,551 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/f77469932e7341a5b296a83235088d80 2023-02-12 04:00:53,559 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/775dbbdc63af49379602accbfda8c0a8 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/775dbbdc63af49379602accbfda8c0a8 2023-02-12 04:00:53,567 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/775dbbdc63af49379602accbfda8c0a8, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:00:53,568 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/f77469932e7341a5b296a83235088d80 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/f77469932e7341a5b296a83235088d80 2023-02-12 04:00:53,576 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/f77469932e7341a5b296a83235088d80, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:00:53,577 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4157975c3727721623be5c93597b743a in 86ms, sequenceid=45, compaction requested=true 2023-02-12 04:00:53,577 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4157975c3727721623be5c93597b743a: 2023-02-12 04:00:53,578 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:53,578 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:00:53,578 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:00:53,578 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,578 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:53,578 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:53,581 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,581 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:53,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:53,581 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,581 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:53,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:53,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:53,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:53,583 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-12 04:00:53,583 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7198164f[Count = 0] remaining members to acquire global barrier 2023-02-12 04:00:53,583 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-12 04:00:53,583 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,584 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,584 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,584 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,584 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:53,584 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:53,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,584 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:00:53,584 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,584 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:00:53,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,584 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:53,584 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:53,584 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:53,584 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:53,584 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:00:53,584 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:00:53,585 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,585 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:53,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:53,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:53,585 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,585 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:53,586 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:53,586 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,586 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:53,587 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:53,587 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:53,587 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:53,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:53,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:00:53,591 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:00:53,591 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-12 04:00:53,591 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:00:53,591 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:00:53,591 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,591 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:00:53,592 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,592 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,592 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,592 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,592 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:53,592 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:53,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,593 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,593 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:53,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,594 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,594 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:53,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:53,596 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,596 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,596 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,598 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,598 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,598 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,598 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,598 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,598 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,598 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,599 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,599 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,598 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,599 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,599 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,599 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-12 04:00:53,599 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:00:53,599 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,599 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,599 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,600 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:00:53,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,600 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:00:53,600 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,601 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,601 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,701 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:00:53,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:00:53,730 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:00:53,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:00:53,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,733 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-12 04:00:53,733 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,734 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-12 04:00:53,734 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-12 04:00:53,735 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,735 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,735 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,735 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,736 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,735 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,736 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,736 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:00:53,737 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,737 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,737 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:53,737 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,737 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,737 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,737 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:53,738 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,738 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:53,738 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:53,738 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,738 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,738 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:53,740 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:53,740 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,739 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:53,740 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:53,740 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:53,740 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:53,740 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,740 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:53,741 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,741 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:00:53,742 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:53,743 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:53,742 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:00:53,743 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. started... 2023-02-12 04:00:53,743 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:53,743 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:53,743 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4157975c3727721623be5c93597b743a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:00:53,753 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,757 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,758 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,758 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:53,758 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,758 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:53,768 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/e8b386936c674e428c100fb6c1e37c29 2023-02-12 04:00:53,799 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/753f49fd0588493bb58f12ab06039bbd 2023-02-12 04:00:53,806 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/e8b386936c674e428c100fb6c1e37c29 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/e8b386936c674e428c100fb6c1e37c29 2023-02-12 04:00:53,813 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/e8b386936c674e428c100fb6c1e37c29, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:00:53,815 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/753f49fd0588493bb58f12ab06039bbd as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/753f49fd0588493bb58f12ab06039bbd 2023-02-12 04:00:53,822 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/753f49fd0588493bb58f12ab06039bbd, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:00:53,823 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4157975c3727721623be5c93597b743a in 80ms, sequenceid=58, compaction requested=true 2023-02-12 04:00:53,824 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4157975c3727721623be5c93597b743a: 2023-02-12 04:00:53,824 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:53,824 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:00:53,825 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:00:53,825 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,825 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:53,825 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:53,826 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,826 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:53,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:53,827 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:53,827 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:53,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:53,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:53,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-12 04:00:53,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43d4e673[Count = 0] remaining members to acquire global barrier 2023-02-12 04:00:53,829 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-12 04:00:53,829 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,831 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,831 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,831 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,831 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:53,831 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,831 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:00:53,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,831 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:53,831 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:53,831 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:53,831 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:53,831 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:53,831 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:00:53,831 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:00:53,831 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:00:53,833 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:53,833 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,833 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:53,833 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:53,833 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,833 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,833 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:53,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:53,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:53,833 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:53,833 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,834 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:53,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:53,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:53,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:53,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:00:53,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:00:53,840 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-12 04:00:53,840 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:00:53,840 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:00:53,840 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,840 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:00:53,848 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,848 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,848 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:53,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:53,848 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,848 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,848 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,849 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:53,848 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,849 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,849 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:53,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,851 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:53,851 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,851 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,857 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,857 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,857 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,857 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,857 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,857 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,857 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,857 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-12 04:00:53,857 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:00:53,857 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:53,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,857 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,857 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:53,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,858 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,858 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:00:53,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,858 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,858 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:00:53,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,858 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:53,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:53,858 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:53,858 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:53,859 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,859 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,859 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,958 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:00:53,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:00:53,986 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:00:53,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:00:53,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,989 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-12 04:00:53,989 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,990 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-12 04:00:53,990 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-12 04:00:53,990 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,991 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,991 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,992 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,992 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,992 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:53,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,992 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:53,992 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:00:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:53,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:53,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:53,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,993 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,993 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,993 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,993 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:53,993 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:53,993 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-12 04:00:53,993 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,994 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,994 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:53,994 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:53,994 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,994 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,994 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,994 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:53,995 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:53,995 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:53,995 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:53,994 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-12 04:00:53,995 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:00:53,995 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:00:53,995 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:53,995 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:53,995 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-12 04:00:53,995 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. started... 2023-02-12 04:00:53,995 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:53,996 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:53,996 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4157975c3727721623be5c93597b743a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:00:53,997 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,998 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,998 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,998 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:53,998 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:53,998 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:54,019 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/2d675530d2de4fbd9a5b012054fd936c 2023-02-12 04:00:54,041 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/5d04d5983ac14c8a8b71b336bffb7cc5 2023-02-12 04:00:54,050 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/2d675530d2de4fbd9a5b012054fd936c as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/2d675530d2de4fbd9a5b012054fd936c 2023-02-12 04:00:54,057 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/2d675530d2de4fbd9a5b012054fd936c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:00:54,058 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/5d04d5983ac14c8a8b71b336bffb7cc5 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/5d04d5983ac14c8a8b71b336bffb7cc5 2023-02-12 04:00:54,066 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/5d04d5983ac14c8a8b71b336bffb7cc5, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:00:54,067 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4157975c3727721623be5c93597b743a in 71ms, sequenceid=71, compaction requested=true 2023-02-12 04:00:54,067 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4157975c3727721623be5c93597b743a: 2023-02-12 04:00:54,068 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:54,068 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:00:54,068 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:00:54,068 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,068 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-12 04:00:54,068 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-12 04:00:54,070 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,070 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:54,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:54,070 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,070 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:54,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:54,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:54,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:54,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:54,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:54,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-12 04:00:54,072 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-12 04:00:54,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5057a743[Count = 0] remaining members to acquire global barrier 2023-02-12 04:00:54,072 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,073 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,073 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,073 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,073 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:54,073 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:54,073 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,073 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:54,073 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:54,073 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:00:54,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,073 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:00:54,073 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:00:54,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,074 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-12 04:00:54,074 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-12 04:00:54,074 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:00:54,074 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:54,074 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:54,074 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:54,075 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:54,075 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,075 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:54,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:54,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:54,075 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:54,076 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-12 04:00:54,076 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:54,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:54,076 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-12 04:00:54,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:54,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:54,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:54,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:54,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:54,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:54,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:00:54,079 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:00:54,079 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-12 04:00:54,079 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:00:54,079 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:00:54,079 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,080 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:00:54,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:54,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:54,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:54,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:54,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,081 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:54,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,081 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:54,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:54,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:54,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:54,082 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:54,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:54,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:54,082 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:54,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:54,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:54,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:54,083 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,084 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,084 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:54,086 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:54,086 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,086 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:54,086 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:54,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:54,086 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:54,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:54,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:54,086 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-12 04:00:54,086 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:00:54,086 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:54,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:54,086 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:54,086 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:54,086 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:54,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:54,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:54,086 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:54,087 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:00:54,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:54,087 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:54,088 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:54,087 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:54,088 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:54,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:54,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:54,087 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:00:54,088 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:54,088 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,088 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,088 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-12 04:00:54,188 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:00:54,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:00:54,215 DEBUG [Listener at localhost.localdomain/35207] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,222 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:54,223 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:54,223 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:54,223 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:54,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4157975c3727721623be5c93597b743a, UNASSIGN}] 2023-02-12 04:00:54,233 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4157975c3727721623be5c93597b743a, UNASSIGN 2023-02-12 04:00:54,234 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=4157975c3727721623be5c93597b743a, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:54,234 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676174454234"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174454234"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174454234"}]},"ts":"1676174454234"} 2023-02-12 04:00:54,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 4157975c3727721623be5c93597b743a, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:00:54,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,397 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 4157975c3727721623be5c93597b743a, disabling compactions & flushes 2023-02-12 04:00:54,397 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:54,397 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:54,397 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. after waiting 0 ms 2023-02-12 04:00:54,397 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:54,397 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 4157975c3727721623be5c93597b743a 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-12 04:00:54,418 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/062c5d3a26aa4ce6bd4348af8e806eb5 2023-02-12 04:00:54,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/2f67d311cad046c5bd6b78f7ea375242 2023-02-12 04:00:54,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf1/062c5d3a26aa4ce6bd4348af8e806eb5 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/062c5d3a26aa4ce6bd4348af8e806eb5 2023-02-12 04:00:54,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/062c5d3a26aa4ce6bd4348af8e806eb5, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:00:54,460 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.tmp/cf2/2f67d311cad046c5bd6b78f7ea375242 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/2f67d311cad046c5bd6b78f7ea375242 2023-02-12 04:00:54,471 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/2f67d311cad046c5bd6b78f7ea375242, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:00:54,472 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4157975c3727721623be5c93597b743a in 75ms, sequenceid=82, compaction requested=true 2023-02-12 04:00:54,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-12 04:00:54,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:00:54,491 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. 2023-02-12 04:00:54,491 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 4157975c3727721623be5c93597b743a: Waiting for close lock at 1676174454397Running coprocessor pre-close hooks at 1676174454397Disabling compacts and flushes for region at 1676174454397Disabling writes for close at 1676174454397Obtaining lock to block concurrent updates at 1676174454397Preparing flush snapshotting stores in 4157975c3727721623be5c93597b743a at 1676174454397Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676174454397Flushing stores of testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. at 1676174454398 (+1 ms)Flushing 4157975c3727721623be5c93597b743a/cf1: creating writer at 1676174454398Flushing 4157975c3727721623be5c93597b743a/cf1: appending metadata at 1676174454401 (+3 ms)Flushing 4157975c3727721623be5c93597b743a/cf1: closing flushed file at 1676174454401Flushing 4157975c3727721623be5c93597b743a/cf2: creating writer at 1676174454426 (+25 ms)Flushing 4157975c3727721623be5c93597b743a/cf2: appending metadata at 1676174454429 (+3 ms)Flushing 4157975c3727721623be5c93597b743a/cf2: closing flushed file at 1676174454429Flushing 4157975c3727721623be5c93597b743a/cf1: reopening flushed file at 1676174454452 (+23 ms)Flushing 4157975c3727721623be5c93597b743a/cf2: reopening flushed file at 1676174454463 (+11 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4157975c3727721623be5c93597b743a in 75ms, sequenceid=82, compaction requested=true at 1676174454472 (+9 ms)Writing region close event to WAL at 1676174454485 (+13 ms)Running coprocessor post-close hooks at 1676174454490 (+5 ms)Closed at 1676174454491 (+1 ms) 2023-02-12 04:00:54,493 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,494 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=4157975c3727721623be5c93597b743a, regionState=CLOSED 2023-02-12 04:00:54,495 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676174454494"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174454494"}]},"ts":"1676174454494"} 2023-02-12 04:00:54,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-12 04:00:54,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 4157975c3727721623be5c93597b743a, server=jenkins-hbase20.apache.org,43977,1676174449184 in 261 msec 2023-02-12 04:00:54,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-12 04:00:54,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4157975c3727721623be5c93597b743a, UNASSIGN in 270 msec 2023-02-12 04:00:54,523 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=4157975c3727721623be5c93597b743a, threads=14 2023-02-12 04:00:54,524 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/2f67d311cad046c5bd6b78f7ea375242 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,524 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/38aef2832cd0444aaabd6f79c51fa0e3 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,524 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/5d04d5983ac14c8a8b71b336bffb7cc5 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,524 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/753f49fd0588493bb58f12ab06039bbd for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,524 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/97c4990de03f488a9deb0213c7a09b8a for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,525 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/b2a62742081b439b85448ce84abdf9d2 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,525 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/f77469932e7341a5b296a83235088d80 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,526 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/062c5d3a26aa4ce6bd4348af8e806eb5 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,526 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/2d675530d2de4fbd9a5b012054fd936c for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,526 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,528 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/6de65c4744c743f794c4fad94c19d8af for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,531 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/775dbbdc63af49379602accbfda8c0a8 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,531 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/d6bf10f37b1c455da3101db2495f96e8 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,533 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/e8b386936c674e428c100fb6c1e37c29 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,556 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/062c5d3a26aa4ce6bd4348af8e806eb5, top=true 2023-02-12 04:00:54,611 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/2f67d311cad046c5bd6b78f7ea375242, top=true 2023-02-12 04:00:54,611 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/5d04d5983ac14c8a8b71b336bffb7cc5, top=true 2023-02-12 04:00:54,616 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/753f49fd0588493bb58f12ab06039bbd, top=true 2023-02-12 04:00:54,617 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/2d675530d2de4fbd9a5b012054fd936c, top=true 2023-02-12 04:00:54,621 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/e8b386936c674e428c100fb6c1e37c29, top=true 2023-02-12 04:00:54,667 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.splits/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2f67d311cad046c5bd6b78f7ea375242 for child: 7cda275d8532d678cc77a56d10e4b57a, parent: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,667 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/2f67d311cad046c5bd6b78f7ea375242 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,670 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.splits/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-753f49fd0588493bb58f12ab06039bbd for child: 7cda275d8532d678cc77a56d10e4b57a, parent: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,675 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/753f49fd0588493bb58f12ab06039bbd for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,679 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.splits/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-5d04d5983ac14c8a8b71b336bffb7cc5 for child: 7cda275d8532d678cc77a56d10e4b57a, parent: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,679 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.splits/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2d675530d2de4fbd9a5b012054fd936c for child: 7cda275d8532d678cc77a56d10e4b57a, parent: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,681 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/2d675530d2de4fbd9a5b012054fd936c for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,681 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/5d04d5983ac14c8a8b71b336bffb7cc5 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,683 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.splits/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-062c5d3a26aa4ce6bd4348af8e806eb5 for child: 7cda275d8532d678cc77a56d10e4b57a, parent: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,683 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/062c5d3a26aa4ce6bd4348af8e806eb5 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,693 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/.splits/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-e8b386936c674e428c100fb6c1e37c29 for child: 7cda275d8532d678cc77a56d10e4b57a, parent: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,694 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/e8b386936c674e428c100fb6c1e37c29 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,747 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/97c4990de03f488a9deb0213c7a09b8a for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,776 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/38aef2832cd0444aaabd6f79c51fa0e3 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,789 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/b2a62742081b439b85448ce84abdf9d2 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,799 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/775dbbdc63af49379602accbfda8c0a8 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,799 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/6de65c4744c743f794c4fad94c19d8af for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,799 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/d6bf10f37b1c455da3101db2495f96e8 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:54,807 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:55,076 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/f77469932e7341a5b296a83235088d80 for region: 4157975c3727721623be5c93597b743a 2023-02-12 04:00:55,077 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 4157975c3727721623be5c93597b743a Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-12 04:00:55,127 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-12 04:00:55,129 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-12 04:00:55,132 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676174455132"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676174455132"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676174455132"}]},"ts":"1676174455132"} 2023-02-12 04:00:55,133 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676174455132"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174455132"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174455132"}]},"ts":"1676174455132"} 2023-02-12 04:00:55,133 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676174455132"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174455132"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174455132"}]},"ts":"1676174455132"} 2023-02-12 04:00:55,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=82e025ba906c703dbab0aa7ffe1c8599, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cda275d8532d678cc77a56d10e4b57a, ASSIGN}] 2023-02-12 04:00:55,178 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cda275d8532d678cc77a56d10e4b57a, ASSIGN 2023-02-12 04:00:55,179 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cda275d8532d678cc77a56d10e4b57a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,43977,1676174449184; forceNewPlan=false, retain=false 2023-02-12 04:00:55,180 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=82e025ba906c703dbab0aa7ffe1c8599, ASSIGN 2023-02-12 04:00:55,181 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=82e025ba906c703dbab0aa7ffe1c8599, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,43977,1676174449184; forceNewPlan=false, retain=false 2023-02-12 04:00:55,329 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-12 04:00:55,331 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=82e025ba906c703dbab0aa7ffe1c8599, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:55,331 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676174455330"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174455330"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174455330"}]},"ts":"1676174455330"} 2023-02-12 04:00:55,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 82e025ba906c703dbab0aa7ffe1c8599, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:00:55,337 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=7cda275d8532d678cc77a56d10e4b57a, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:55,337 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676174455337"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174455337"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174455337"}]},"ts":"1676174455337"} 2023-02-12 04:00:55,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 7cda275d8532d678cc77a56d10e4b57a, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:00:55,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a. 2023-02-12 04:00:55,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7cda275d8532d678cc77a56d10e4b57a, NAME => 'testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.', STARTKEY => '26', ENDKEY => ''} 2023-02-12 04:00:55,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:00:55,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:55,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:00:55,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:55,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:55,492 INFO [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:55,493 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1 2023-02-12 04:00:55,493 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1 2023-02-12 04:00:55,494 INFO [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7cda275d8532d678cc77a56d10e4b57a columnFamilyName cf1 2023-02-12 04:00:55,527 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3-top 2023-02-12 04:00:55,534 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/6de65c4744c743f794c4fad94c19d8af.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/6de65c4744c743f794c4fad94c19d8af-top 2023-02-12 04:00:55,543 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/775dbbdc63af49379602accbfda8c0a8.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/775dbbdc63af49379602accbfda8c0a8-top 2023-02-12 04:00:55,550 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/d6bf10f37b1c455da3101db2495f96e8.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/d6bf10f37b1c455da3101db2495f96e8-top 2023-02-12 04:00:55,557 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-062c5d3a26aa4ce6bd4348af8e806eb5 2023-02-12 04:00:55,563 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2d675530d2de4fbd9a5b012054fd936c 2023-02-12 04:00:55,569 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-e8b386936c674e428c100fb6c1e37c29 2023-02-12 04:00:55,570 INFO [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(310): Store=7cda275d8532d678cc77a56d10e4b57a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:55,570 INFO [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:55,571 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2 2023-02-12 04:00:55,572 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2 2023-02-12 04:00:55,572 INFO [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7cda275d8532d678cc77a56d10e4b57a columnFamilyName cf2 2023-02-12 04:00:55,588 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/38aef2832cd0444aaabd6f79c51fa0e3.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/38aef2832cd0444aaabd6f79c51fa0e3-top 2023-02-12 04:00:55,594 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/97c4990de03f488a9deb0213c7a09b8a.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/97c4990de03f488a9deb0213c7a09b8a-top 2023-02-12 04:00:55,600 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/b2a62742081b439b85448ce84abdf9d2.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/b2a62742081b439b85448ce84abdf9d2-top 2023-02-12 04:00:55,605 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/f77469932e7341a5b296a83235088d80.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/f77469932e7341a5b296a83235088d80-top 2023-02-12 04:00:55,610 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2f67d311cad046c5bd6b78f7ea375242 2023-02-12 04:00:55,616 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-5d04d5983ac14c8a8b71b336bffb7cc5 2023-02-12 04:00:55,622 DEBUG [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-753f49fd0588493bb58f12ab06039bbd 2023-02-12 04:00:55,623 INFO [StoreOpener-7cda275d8532d678cc77a56d10e4b57a-1] regionserver.HStore(310): Store=7cda275d8532d678cc77a56d10e4b57a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:55,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:55,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:55,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:00:55,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:55,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 7cda275d8532d678cc77a56d10e4b57a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74447350, jitterRate=0.10935196280479431}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:00:55,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 7cda275d8532d678cc77a56d10e4b57a: Running coprocessor pre-open hook at 1676174455491Writing region info on filesystem at 1676174455491Initializing all the Stores at 1676174455492 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174455492Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174455492Cleaning up temporary data from old regions at 1676174455627 (+135 ms)Cleaning up detritus from prior splits at 1676174455627Running coprocessor post-open hooks at 1676174455630 (+3 ms)Region opened successfully at 1676174455630 2023-02-12 04:00:55,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a., pid=18, masterSystemTime=1676174455486 2023-02-12 04:00:55,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:00:55,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:00:55,634 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:00:55,634 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:00:55,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a. 2023-02-12 04:00:55,637 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a. 2023-02-12 04:00:55,637 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a. 2023-02-12 04:00:55,637 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1912): 7cda275d8532d678cc77a56d10e4b57a/cf1 is initiating minor compaction (all files) 2023-02-12 04:00:55,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a. 2023-02-12 04:00:55,638 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1912): 7cda275d8532d678cc77a56d10e4b57a/cf2 is initiating minor compaction (all files) 2023-02-12 04:00:55,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599. 2023-02-12 04:00:55,638 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 7cda275d8532d678cc77a56d10e4b57a/cf2 in testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a. 2023-02-12 04:00:55,638 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7cda275d8532d678cc77a56d10e4b57a/cf1 in testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a. 2023-02-12 04:00:55,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 82e025ba906c703dbab0aa7ffe1c8599, NAME => 'testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.', STARTKEY => '', ENDKEY => '26'} 2023-02-12 04:00:55,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:00:55,638 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/97c4990de03f488a9deb0213c7a09b8a.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/97c4990de03f488a9deb0213c7a09b8a-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/38aef2832cd0444aaabd6f79c51fa0e3.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/38aef2832cd0444aaabd6f79c51fa0e3-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/b2a62742081b439b85448ce84abdf9d2.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/b2a62742081b439b85448ce84abdf9d2-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/f77469932e7341a5b296a83235088d80.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/f77469932e7341a5b296a83235088d80-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-753f49fd0588493bb58f12ab06039bbd, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-5d04d5983ac14c8a8b71b336bffb7cc5, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2f67d311cad046c5bd6b78f7ea375242] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/.tmp, totalSize=34.9 K 2023-02-12 04:00:55,638 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=7cda275d8532d678cc77a56d10e4b57a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:55,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 82e025ba906c703dbab0aa7ffe1c8599 2023-02-12 04:00:55,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:00:55,639 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676174455638"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174455638"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174455638"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174455638"}]},"ts":"1676174455638"} 2023-02-12 04:00:55,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 82e025ba906c703dbab0aa7ffe1c8599 2023-02-12 04:00:55,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 82e025ba906c703dbab0aa7ffe1c8599 2023-02-12 04:00:55,638 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/6de65c4744c743f794c4fad94c19d8af.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/6de65c4744c743f794c4fad94c19d8af-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/d6bf10f37b1c455da3101db2495f96e8.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/d6bf10f37b1c455da3101db2495f96e8-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/775dbbdc63af49379602accbfda8c0a8.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/775dbbdc63af49379602accbfda8c0a8-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-e8b386936c674e428c100fb6c1e37c29, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2d675530d2de4fbd9a5b012054fd936c, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-062c5d3a26aa4ce6bd4348af8e806eb5] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/.tmp, totalSize=34.9 K 2023-02-12 04:00:55,642 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 97c4990de03f488a9deb0213c7a09b8a.4157975c3727721623be5c93597b743a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676174452557 2023-02-12 04:00:55,642 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 6de65c4744c743f794c4fad94c19d8af.4157975c3727721623be5c93597b743a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676174452557 2023-02-12 04:00:55,643 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 38aef2832cd0444aaabd6f79c51fa0e3.4157975c3727721623be5c93597b743a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676174452940 2023-02-12 04:00:55,644 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting d6bf10f37b1c455da3101db2495f96e8.4157975c3727721623be5c93597b743a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676174452940 2023-02-12 04:00:55,645 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting b2a62742081b439b85448ce84abdf9d2.4157975c3727721623be5c93597b743a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676174453199 2023-02-12 04:00:55,645 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting f77469932e7341a5b296a83235088d80.4157975c3727721623be5c93597b743a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676174453445 2023-02-12 04:00:55,646 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 1f1eeaebaf1a4d19843eb4973ba0aaa3.4157975c3727721623be5c93597b743a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676174453199 2023-02-12 04:00:55,646 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 775dbbdc63af49379602accbfda8c0a8.4157975c3727721623be5c93597b743a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676174453445 2023-02-12 04:00:55,647 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-753f49fd0588493bb58f12ab06039bbd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174453703 2023-02-12 04:00:55,647 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-e8b386936c674e428c100fb6c1e37c29, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174453703 2023-02-12 04:00:55,647 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-5d04d5983ac14c8a8b71b336bffb7cc5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174453961 2023-02-12 04:00:55,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-02-12 04:00:55,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 7cda275d8532d678cc77a56d10e4b57a, server=jenkins-hbase20.apache.org,43977,1676174449184 in 305 msec 2023-02-12 04:00:55,648 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2d675530d2de4fbd9a5b012054fd936c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174453961 2023-02-12 04:00:55,648 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2f67d311cad046c5bd6b78f7ea375242, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676174454191 2023-02-12 04:00:55,649 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-062c5d3a26aa4ce6bd4348af8e806eb5, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676174454191 2023-02-12 04:00:55,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cda275d8532d678cc77a56d10e4b57a, ASSIGN in 472 msec 2023-02-12 04:00:55,656 INFO [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 82e025ba906c703dbab0aa7ffe1c8599 2023-02-12 04:00:55,658 DEBUG [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1 2023-02-12 04:00:55,658 DEBUG [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1 2023-02-12 04:00:55,658 INFO [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 82e025ba906c703dbab0aa7ffe1c8599 columnFamilyName cf1 2023-02-12 04:00:55,688 DEBUG [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3-bottom 2023-02-12 04:00:55,694 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7cda275d8532d678cc77a56d10e4b57a#cf1#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:00:55,697 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] throttle.PressureAwareThroughputController(145): 7cda275d8532d678cc77a56d10e4b57a#cf2#compaction#15 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:00:55,707 DEBUG [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/6de65c4744c743f794c4fad94c19d8af.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/6de65c4744c743f794c4fad94c19d8af-bottom 2023-02-12 04:00:55,712 DEBUG [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/775dbbdc63af49379602accbfda8c0a8.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/775dbbdc63af49379602accbfda8c0a8-bottom 2023-02-12 04:00:55,718 DEBUG [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/d6bf10f37b1c455da3101db2495f96e8.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/d6bf10f37b1c455da3101db2495f96e8-bottom 2023-02-12 04:00:55,728 INFO [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] regionserver.HStore(310): Store=82e025ba906c703dbab0aa7ffe1c8599/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:55,728 INFO [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 82e025ba906c703dbab0aa7ffe1c8599 2023-02-12 04:00:55,730 DEBUG [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2 2023-02-12 04:00:55,730 DEBUG [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2 2023-02-12 04:00:55,730 INFO [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 82e025ba906c703dbab0aa7ffe1c8599 columnFamilyName cf2 2023-02-12 04:00:55,736 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/.tmp/cf1/49a8973d45ba487a94a6dbb72322ade9 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/49a8973d45ba487a94a6dbb72322ade9 2023-02-12 04:00:55,753 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7cda275d8532d678cc77a56d10e4b57a/cf1 of 7cda275d8532d678cc77a56d10e4b57a into 49a8973d45ba487a94a6dbb72322ade9(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:00:55,754 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7cda275d8532d678cc77a56d10e4b57a: 2023-02-12 04:00:55,754 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a., storeName=7cda275d8532d678cc77a56d10e4b57a/cf1, priority=9, startTime=1676174455633; duration=0sec 2023-02-12 04:00:55,755 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:00:55,755 DEBUG [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/38aef2832cd0444aaabd6f79c51fa0e3.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/38aef2832cd0444aaabd6f79c51fa0e3-bottom 2023-02-12 04:00:55,760 DEBUG [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/97c4990de03f488a9deb0213c7a09b8a.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/97c4990de03f488a9deb0213c7a09b8a-bottom 2023-02-12 04:00:55,761 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/.tmp/cf2/267e98ac81804d4aaf3eb46379fe32af as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/267e98ac81804d4aaf3eb46379fe32af 2023-02-12 04:00:55,767 DEBUG [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/b2a62742081b439b85448ce84abdf9d2.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/b2a62742081b439b85448ce84abdf9d2-bottom 2023-02-12 04:00:55,770 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7cda275d8532d678cc77a56d10e4b57a/cf2 of 7cda275d8532d678cc77a56d10e4b57a into 267e98ac81804d4aaf3eb46379fe32af(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:00:55,770 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7cda275d8532d678cc77a56d10e4b57a: 2023-02-12 04:00:55,770 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a., storeName=7cda275d8532d678cc77a56d10e4b57a/cf2, priority=9, startTime=1676174455634; duration=0sec 2023-02-12 04:00:55,770 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:00:55,773 DEBUG [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/f77469932e7341a5b296a83235088d80.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/f77469932e7341a5b296a83235088d80-bottom 2023-02-12 04:00:55,774 INFO [StoreOpener-82e025ba906c703dbab0aa7ffe1c8599-1] regionserver.HStore(310): Store=82e025ba906c703dbab0aa7ffe1c8599/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:55,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599 2023-02-12 04:00:55,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599 2023-02-12 04:00:55,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:00:55,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 82e025ba906c703dbab0aa7ffe1c8599 2023-02-12 04:00:55,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 82e025ba906c703dbab0aa7ffe1c8599; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63643332, jitterRate=-0.051640450954437256}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:00:55,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 82e025ba906c703dbab0aa7ffe1c8599: Running coprocessor pre-open hook at 1676174455640Writing region info on filesystem at 1676174455640Initializing all the Stores at 1676174455642 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174455642Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174455655 (+13 ms)Cleaning up temporary data from old regions at 1676174455777 (+122 ms)Cleaning up detritus from prior splits at 1676174455777Running coprocessor post-open hooks at 1676174455780 (+3 ms)Region opened successfully at 1676174455780 2023-02-12 04:00:55,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599., pid=17, masterSystemTime=1676174455486 2023-02-12 04:00:55,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:00:55,782 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-12 04:00:55,782 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-12 04:00:55,782 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599. 2023-02-12 04:00:55,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:00:55,782 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1912): 82e025ba906c703dbab0aa7ffe1c8599/cf1 is initiating minor compaction (all files) 2023-02-12 04:00:55,782 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 82e025ba906c703dbab0aa7ffe1c8599/cf1 in testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599. 2023-02-12 04:00:55,782 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599. 2023-02-12 04:00:55,783 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1912): 82e025ba906c703dbab0aa7ffe1c8599/cf2 is initiating minor compaction (all files) 2023-02-12 04:00:55,783 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 82e025ba906c703dbab0aa7ffe1c8599/cf2 in testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599. 2023-02-12 04:00:55,783 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/6de65c4744c743f794c4fad94c19d8af.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/6de65c4744c743f794c4fad94c19d8af-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/d6bf10f37b1c455da3101db2495f96e8.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/d6bf10f37b1c455da3101db2495f96e8-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/775dbbdc63af49379602accbfda8c0a8.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/775dbbdc63af49379602accbfda8c0a8-bottom] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/.tmp, totalSize=19.9 K 2023-02-12 04:00:55,783 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/97c4990de03f488a9deb0213c7a09b8a.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/97c4990de03f488a9deb0213c7a09b8a-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/38aef2832cd0444aaabd6f79c51fa0e3.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/38aef2832cd0444aaabd6f79c51fa0e3-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/b2a62742081b439b85448ce84abdf9d2.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/b2a62742081b439b85448ce84abdf9d2-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/f77469932e7341a5b296a83235088d80.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/f77469932e7341a5b296a83235088d80-bottom] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/.tmp, totalSize=19.9 K 2023-02-12 04:00:55,784 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 6de65c4744c743f794c4fad94c19d8af.4157975c3727721623be5c93597b743a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174452557 2023-02-12 04:00:55,784 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 97c4990de03f488a9deb0213c7a09b8a.4157975c3727721623be5c93597b743a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174452557 2023-02-12 04:00:55,785 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting d6bf10f37b1c455da3101db2495f96e8.4157975c3727721623be5c93597b743a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174452940 2023-02-12 04:00:55,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599. 2023-02-12 04:00:55,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599. 2023-02-12 04:00:55,787 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 1f1eeaebaf1a4d19843eb4973ba0aaa3.4157975c3727721623be5c93597b743a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174453199 2023-02-12 04:00:55,787 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=82e025ba906c703dbab0aa7ffe1c8599, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:55,787 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676174455787"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174455787"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174455787"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174455787"}]},"ts":"1676174455787"} 2023-02-12 04:00:55,788 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 775dbbdc63af49379602accbfda8c0a8.4157975c3727721623be5c93597b743a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174453445 2023-02-12 04:00:55,788 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 38aef2832cd0444aaabd6f79c51fa0e3.4157975c3727721623be5c93597b743a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174452940 2023-02-12 04:00:55,789 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting b2a62742081b439b85448ce84abdf9d2.4157975c3727721623be5c93597b743a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174453199 2023-02-12 04:00:55,790 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting f77469932e7341a5b296a83235088d80.4157975c3727721623be5c93597b743a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174453445 2023-02-12 04:00:55,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-12 04:00:55,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 82e025ba906c703dbab0aa7ffe1c8599, server=jenkins-hbase20.apache.org,43977,1676174449184 in 457 msec 2023-02-12 04:00:55,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-02-12 04:00:55,801 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=82e025ba906c703dbab0aa7ffe1c8599, ASSIGN in 620 msec 2023-02-12 04:00:55,805 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a in 1.5860 sec 2023-02-12 04:00:55,809 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] throttle.PressureAwareThroughputController(145): 82e025ba906c703dbab0aa7ffe1c8599#cf1#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:00:55,812 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] throttle.PressureAwareThroughputController(145): 82e025ba906c703dbab0aa7ffe1c8599#cf2#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:00:55,845 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/.tmp/cf1/1a5c010d9ec2411f90352a66ce8d8166 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/1a5c010d9ec2411f90352a66ce8d8166 2023-02-12 04:00:55,849 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/.tmp/cf2/501bb3b2e26b427485b6b93cd4ca90f6 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/501bb3b2e26b427485b6b93cd4ca90f6 2023-02-12 04:00:55,854 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 82e025ba906c703dbab0aa7ffe1c8599/cf1 of 82e025ba906c703dbab0aa7ffe1c8599 into 1a5c010d9ec2411f90352a66ce8d8166(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:00:55,854 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 82e025ba906c703dbab0aa7ffe1c8599: 2023-02-12 04:00:55,854 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599., storeName=82e025ba906c703dbab0aa7ffe1c8599/cf1, priority=12, startTime=1676174455781; duration=0sec 2023-02-12 04:00:55,854 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:00:55,857 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 82e025ba906c703dbab0aa7ffe1c8599/cf2 of 82e025ba906c703dbab0aa7ffe1c8599 into 501bb3b2e26b427485b6b93cd4ca90f6(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:00:55,857 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 82e025ba906c703dbab0aa7ffe1c8599: 2023-02-12 04:00:55,857 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599., storeName=82e025ba906c703dbab0aa7ffe1c8599/cf2, priority=12, startTime=1676174455782; duration=0sec 2023-02-12 04:00:55,857 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:00:55,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43977] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 148.251.75.209:52638 deadline: 1676174515974, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a. is not online on jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:56,106 DEBUG [Listener at localhost.localdomain/35207] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-12 04:00:56,121 INFO [Listener at localhost.localdomain/35207] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-12 04:00:56,125 DEBUG [Listener at localhost.localdomain/35207] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-12 04:00:56,159 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:00:56,171 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-12 04:00:56,176 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testSplitTableRegionUnevenDaughter 2023-02-12 04:00:56,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:00:56,187 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174456187"}]},"ts":"1676174456187"} 2023-02-12 04:00:56,189 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-12 04:00:56,191 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-12 04:00:56,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=82e025ba906c703dbab0aa7ffe1c8599, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cda275d8532d678cc77a56d10e4b57a, UNASSIGN}] 2023-02-12 04:00:56,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-12 04:00:56,195 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cda275d8532d678cc77a56d10e4b57a, UNASSIGN 2023-02-12 04:00:56,196 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7cda275d8532d678cc77a56d10e4b57a, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:56,196 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676174456196"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174456196"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174456196"}]},"ts":"1676174456196"} 2023-02-12 04:00:56,199 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 7cda275d8532d678cc77a56d10e4b57a, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:00:56,204 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=82e025ba906c703dbab0aa7ffe1c8599, UNASSIGN 2023-02-12 04:00:56,205 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=82e025ba906c703dbab0aa7ffe1c8599, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:56,205 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676174456205"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174456205"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174456205"}]},"ts":"1676174456205"} 2023-02-12 04:00:56,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 82e025ba906c703dbab0aa7ffe1c8599, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:00:56,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-12 04:00:56,352 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 82e025ba906c703dbab0aa7ffe1c8599 2023-02-12 04:00:56,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 82e025ba906c703dbab0aa7ffe1c8599, disabling compactions & flushes 2023-02-12 04:00:56,353 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599. 2023-02-12 04:00:56,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599. 2023-02-12 04:00:56,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599. after waiting 0 ms 2023-02-12 04:00:56,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599. 2023-02-12 04:00:56,358 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/6de65c4744c743f794c4fad94c19d8af.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/6de65c4744c743f794c4fad94c19d8af-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/d6bf10f37b1c455da3101db2495f96e8.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/d6bf10f37b1c455da3101db2495f96e8-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/775dbbdc63af49379602accbfda8c0a8.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/775dbbdc63af49379602accbfda8c0a8-bottom] to archive 2023-02-12 04:00:56,359 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:00:56,366 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/6de65c4744c743f794c4fad94c19d8af.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/6de65c4744c743f794c4fad94c19d8af.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,370 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/d6bf10f37b1c455da3101db2495f96e8.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/d6bf10f37b1c455da3101db2495f96e8.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,373 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,375 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/775dbbdc63af49379602accbfda8c0a8.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/775dbbdc63af49379602accbfda8c0a8.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,403 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/97c4990de03f488a9deb0213c7a09b8a.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/97c4990de03f488a9deb0213c7a09b8a-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/38aef2832cd0444aaabd6f79c51fa0e3.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/38aef2832cd0444aaabd6f79c51fa0e3-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/b2a62742081b439b85448ce84abdf9d2.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/b2a62742081b439b85448ce84abdf9d2-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/f77469932e7341a5b296a83235088d80.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/f77469932e7341a5b296a83235088d80-bottom] to archive 2023-02-12 04:00:56,406 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:00:56,408 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/97c4990de03f488a9deb0213c7a09b8a.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/97c4990de03f488a9deb0213c7a09b8a.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,412 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/38aef2832cd0444aaabd6f79c51fa0e3.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/38aef2832cd0444aaabd6f79c51fa0e3.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,414 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/b2a62742081b439b85448ce84abdf9d2.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/b2a62742081b439b85448ce84abdf9d2.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,416 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/f77469932e7341a5b296a83235088d80.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/f77469932e7341a5b296a83235088d80.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,426 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-12 04:00:56,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:00:56,427 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599. 2023-02-12 04:00:56,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 82e025ba906c703dbab0aa7ffe1c8599: Waiting for close lock at 1676174456352Running coprocessor pre-close hooks at 1676174456352Disabling compacts and flushes for region at 1676174456352Disabling writes for close at 1676174456353 (+1 ms)Writing region close event to WAL at 1676174456419 (+66 ms)Running coprocessor post-close hooks at 1676174456427 (+8 ms)Closed at 1676174456427 2023-02-12 04:00:56,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 82e025ba906c703dbab0aa7ffe1c8599 2023-02-12 04:00:56,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:56,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 7cda275d8532d678cc77a56d10e4b57a, disabling compactions & flushes 2023-02-12 04:00:56,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a. 2023-02-12 04:00:56,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a. 2023-02-12 04:00:56,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a. after waiting 0 ms 2023-02-12 04:00:56,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a. 2023-02-12 04:00:56,431 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=82e025ba906c703dbab0aa7ffe1c8599, regionState=CLOSED 2023-02-12 04:00:56,431 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676174456431"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174456431"}]},"ts":"1676174456431"} 2023-02-12 04:00:56,434 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/6de65c4744c743f794c4fad94c19d8af.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/6de65c4744c743f794c4fad94c19d8af-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/d6bf10f37b1c455da3101db2495f96e8.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/d6bf10f37b1c455da3101db2495f96e8-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/775dbbdc63af49379602accbfda8c0a8.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/775dbbdc63af49379602accbfda8c0a8-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-e8b386936c674e428c100fb6c1e37c29, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2d675530d2de4fbd9a5b012054fd936c, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-062c5d3a26aa4ce6bd4348af8e806eb5] to archive 2023-02-12 04:00:56,435 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:00:56,439 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/6de65c4744c743f794c4fad94c19d8af.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/6de65c4744c743f794c4fad94c19d8af.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-12 04:00:56,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 82e025ba906c703dbab0aa7ffe1c8599, server=jenkins-hbase20.apache.org,43977,1676174449184 in 226 msec 2023-02-12 04:00:56,442 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/d6bf10f37b1c455da3101db2495f96e8.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/d6bf10f37b1c455da3101db2495f96e8.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=82e025ba906c703dbab0aa7ffe1c8599, UNASSIGN in 246 msec 2023-02-12 04:00:56,444 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,445 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/775dbbdc63af49379602accbfda8c0a8.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/775dbbdc63af49379602accbfda8c0a8.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,447 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-e8b386936c674e428c100fb6c1e37c29 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-e8b386936c674e428c100fb6c1e37c29 2023-02-12 04:00:56,449 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2d675530d2de4fbd9a5b012054fd936c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2d675530d2de4fbd9a5b012054fd936c 2023-02-12 04:00:56,451 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-062c5d3a26aa4ce6bd4348af8e806eb5 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-062c5d3a26aa4ce6bd4348af8e806eb5 2023-02-12 04:00:56,453 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/97c4990de03f488a9deb0213c7a09b8a.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/97c4990de03f488a9deb0213c7a09b8a-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/38aef2832cd0444aaabd6f79c51fa0e3.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/38aef2832cd0444aaabd6f79c51fa0e3-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/b2a62742081b439b85448ce84abdf9d2.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/b2a62742081b439b85448ce84abdf9d2-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/f77469932e7341a5b296a83235088d80.4157975c3727721623be5c93597b743a->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/f77469932e7341a5b296a83235088d80-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-753f49fd0588493bb58f12ab06039bbd, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-5d04d5983ac14c8a8b71b336bffb7cc5, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2f67d311cad046c5bd6b78f7ea375242] to archive 2023-02-12 04:00:56,455 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:00:56,458 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/97c4990de03f488a9deb0213c7a09b8a.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/97c4990de03f488a9deb0213c7a09b8a.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,460 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/38aef2832cd0444aaabd6f79c51fa0e3.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/38aef2832cd0444aaabd6f79c51fa0e3.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,462 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/b2a62742081b439b85448ce84abdf9d2.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/b2a62742081b439b85448ce84abdf9d2.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,463 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/f77469932e7341a5b296a83235088d80.4157975c3727721623be5c93597b743a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/f77469932e7341a5b296a83235088d80.4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,465 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-753f49fd0588493bb58f12ab06039bbd to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-753f49fd0588493bb58f12ab06039bbd 2023-02-12 04:00:56,467 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-5d04d5983ac14c8a8b71b336bffb7cc5 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-5d04d5983ac14c8a8b71b336bffb7cc5 2023-02-12 04:00:56,468 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2f67d311cad046c5bd6b78f7ea375242 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/testSplitTableRegionUnevenDaughter=4157975c3727721623be5c93597b743a-2f67d311cad046c5bd6b78f7ea375242 2023-02-12 04:00:56,474 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 04:00:56,477 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-12 04:00:56,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:00:56,479 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a. 2023-02-12 04:00:56,479 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 7cda275d8532d678cc77a56d10e4b57a: Waiting for close lock at 1676174456430Running coprocessor pre-close hooks at 1676174456430Disabling compacts and flushes for region at 1676174456430Disabling writes for close at 1676174456430Writing region close event to WAL at 1676174456470 (+40 ms)Running coprocessor post-close hooks at 1676174456478 (+8 ms)Closed at 1676174456479 (+1 ms) 2023-02-12 04:00:56,481 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:56,482 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7cda275d8532d678cc77a56d10e4b57a, regionState=CLOSED 2023-02-12 04:00:56,483 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676174456482"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174456482"}]},"ts":"1676174456482"} 2023-02-12 04:00:56,488 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-12 04:00:56,488 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 7cda275d8532d678cc77a56d10e4b57a, server=jenkins-hbase20.apache.org,43977,1676174449184 in 286 msec 2023-02-12 04:00:56,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-02-12 04:00:56,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7cda275d8532d678cc77a56d10e4b57a, UNASSIGN in 296 msec 2023-02-12 04:00:56,494 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174456494"}]},"ts":"1676174456494"} 2023-02-12 04:00:56,496 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-12 04:00:56,497 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-12 04:00:56,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-12 04:00:56,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 321 msec 2023-02-12 04:00:56,561 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-12 04:00:56,561 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-12 04:00:56,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-12 04:00:56,802 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-12 04:00:56,807 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testSplitTableRegionUnevenDaughter 2023-02-12 04:00:56,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:00:56,815 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:00:56,818 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:00:56,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-12 04:00:56,826 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,826 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599 2023-02-12 04:00:56,826 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:56,830 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/recovered.edits] 2023-02-12 04:00:56,830 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/recovered.edits] 2023-02-12 04:00:56,830 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/recovered.edits] 2023-02-12 04:00:56,834 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/49a8973d45ba487a94a6dbb72322ade9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf1/49a8973d45ba487a94a6dbb72322ade9 2023-02-12 04:00:56,834 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/062c5d3a26aa4ce6bd4348af8e806eb5 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/062c5d3a26aa4ce6bd4348af8e806eb5 2023-02-12 04:00:56,834 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/1a5c010d9ec2411f90352a66ce8d8166 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf1/1a5c010d9ec2411f90352a66ce8d8166 2023-02-12 04:00:56,837 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/1f1eeaebaf1a4d19843eb4973ba0aaa3 2023-02-12 04:00:56,837 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/267e98ac81804d4aaf3eb46379fe32af to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/cf2/267e98ac81804d4aaf3eb46379fe32af 2023-02-12 04:00:56,838 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/501bb3b2e26b427485b6b93cd4ca90f6 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/cf2/501bb3b2e26b427485b6b93cd4ca90f6 2023-02-12 04:00:56,843 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/2d675530d2de4fbd9a5b012054fd936c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/2d675530d2de4fbd9a5b012054fd936c 2023-02-12 04:00:56,844 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/recovered.edits/90.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a/recovered.edits/90.seqid 2023-02-12 04:00:56,845 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/6de65c4744c743f794c4fad94c19d8af to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/6de65c4744c743f794c4fad94c19d8af 2023-02-12 04:00:56,845 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/recovered.edits/90.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599/recovered.edits/90.seqid 2023-02-12 04:00:56,845 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:56,846 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/82e025ba906c703dbab0aa7ffe1c8599 2023-02-12 04:00:56,847 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/775dbbdc63af49379602accbfda8c0a8 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/775dbbdc63af49379602accbfda8c0a8 2023-02-12 04:00:56,848 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/d6bf10f37b1c455da3101db2495f96e8 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/d6bf10f37b1c455da3101db2495f96e8 2023-02-12 04:00:56,849 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/e8b386936c674e428c100fb6c1e37c29 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf1/e8b386936c674e428c100fb6c1e37c29 2023-02-12 04:00:56,852 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/2f67d311cad046c5bd6b78f7ea375242 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/2f67d311cad046c5bd6b78f7ea375242 2023-02-12 04:00:56,854 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/38aef2832cd0444aaabd6f79c51fa0e3 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/38aef2832cd0444aaabd6f79c51fa0e3 2023-02-12 04:00:56,855 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/5d04d5983ac14c8a8b71b336bffb7cc5 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/5d04d5983ac14c8a8b71b336bffb7cc5 2023-02-12 04:00:56,856 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/753f49fd0588493bb58f12ab06039bbd to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/753f49fd0588493bb58f12ab06039bbd 2023-02-12 04:00:56,858 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/97c4990de03f488a9deb0213c7a09b8a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/97c4990de03f488a9deb0213c7a09b8a 2023-02-12 04:00:56,859 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/b2a62742081b439b85448ce84abdf9d2 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/b2a62742081b439b85448ce84abdf9d2 2023-02-12 04:00:56,861 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/f77469932e7341a5b296a83235088d80 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/cf2/f77469932e7341a5b296a83235088d80 2023-02-12 04:00:56,865 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/recovered.edits/85.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a/recovered.edits/85.seqid 2023-02-12 04:00:56,865 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionUnevenDaughter/4157975c3727721623be5c93597b743a 2023-02-12 04:00:56,865 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-12 04:00:56,868 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:00:56,887 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-12 04:00:56,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-12 04:00:56,930 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-12 04:00:56,932 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:00:56,932 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-12 04:00:56,932 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174456932"}]},"ts":"9223372036854775807"} 2023-02-12 04:00:56,932 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174456932"}]},"ts":"9223372036854775807"} 2023-02-12 04:00:56,932 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174456932"}]},"ts":"9223372036854775807"} 2023-02-12 04:00:56,936 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-12 04:00:56,936 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4157975c3727721623be5c93597b743a, NAME => 'testSplitTableRegionUnevenDaughter,,1676174452049.4157975c3727721623be5c93597b743a.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 82e025ba906c703dbab0aa7ffe1c8599, NAME => 'testSplitTableRegionUnevenDaughter,,1676174454215.82e025ba906c703dbab0aa7ffe1c8599.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 7cda275d8532d678cc77a56d10e4b57a, NAME => 'testSplitTableRegionUnevenDaughter,26,1676174454215.7cda275d8532d678cc77a56d10e4b57a.', STARTKEY => '26', ENDKEY => ''}] 2023-02-12 04:00:56,936 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-12 04:00:56,936 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676174456936"}]},"ts":"9223372036854775807"} 2023-02-12 04:00:56,938 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-12 04:00:56,940 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:00:56,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 133 msec 2023-02-12 04:00:57,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-12 04:00:57,122 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-12 04:00:57,136 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=389 (was 358) Potentially hanging thread: hconnection-0x6c2e965c-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:38250 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:38494 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c2e965c-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:59198 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:56176 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:56152 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase20:43977-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c2e965c-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:56248 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:39804 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:56200 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:37420 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:37530 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:40044 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c2e965c-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c2e965c-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:40096 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:37542 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:53110 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:52954 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:38278 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=844 (was 691) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=204 (was 213), ProcessCount=165 (was 165), AvailableMemoryMB=4280 (was 4326) 2023-02-12 04:00:57,149 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=389, OpenFileDescriptor=844, MaxFileDescriptor=60000, SystemLoadAverage=204, ProcessCount=165, AvailableMemoryMB=4279 2023-02-12 04:00:57,150 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:00:57,154 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-12 04:00:57,154 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-12 04:00:57,156 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:00:57,158 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 04:00:57,159 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 04:00:57,160 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:00:57,161 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f empty. 2023-02-12 04:00:57,161 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:00:57,161 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-12 04:00:57,173 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-12 04:00:57,174 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 14a5b26b7821b29d8e7405e41ed6325f, NAME => 'testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp 2023-02-12 04:00:57,188 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:00:57,188 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 14a5b26b7821b29d8e7405e41ed6325f, disabling compactions & flushes 2023-02-12 04:00:57,188 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:57,188 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:57,188 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. after waiting 0 ms 2023-02-12 04:00:57,188 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:57,188 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:57,188 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 14a5b26b7821b29d8e7405e41ed6325f: Waiting for close lock at 1676174457188Disabling compacts and flushes for region at 1676174457188Disabling writes for close at 1676174457188Writing region close event to WAL at 1676174457188Closed at 1676174457188 2023-02-12 04:00:57,191 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 04:00:57,192 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174457192"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174457192"}]},"ts":"1676174457192"} 2023-02-12 04:00:57,195 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 04:00:57,196 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 04:00:57,196 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174457196"}]},"ts":"1676174457196"} 2023-02-12 04:00:57,198 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-12 04:00:57,200 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-12 04:00:57,201 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-12 04:00:57,201 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-12 04:00:57,201 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-12 04:00:57,201 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-12 04:00:57,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, ASSIGN}] 2023-02-12 04:00:57,204 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, ASSIGN 2023-02-12 04:00:57,204 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,43977,1676174449184; forceNewPlan=false, retain=false 2023-02-12 04:00:57,355 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-12 04:00:57,356 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=14a5b26b7821b29d8e7405e41ed6325f, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,356 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174457356"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174457356"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174457356"}]},"ts":"1676174457356"} 2023-02-12 04:00:57,358 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:00:57,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:57,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 14a5b26b7821b29d8e7405e41ed6325f, NAME => 'testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:00:57,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:00:57,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:00:57,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:00:57,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:00:57,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:00:57,516 INFO [StoreOpener-14a5b26b7821b29d8e7405e41ed6325f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:00:57,518 DEBUG [StoreOpener-14a5b26b7821b29d8e7405e41ed6325f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1 2023-02-12 04:00:57,518 DEBUG [StoreOpener-14a5b26b7821b29d8e7405e41ed6325f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1 2023-02-12 04:00:57,518 INFO [StoreOpener-14a5b26b7821b29d8e7405e41ed6325f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 14a5b26b7821b29d8e7405e41ed6325f columnFamilyName cf1 2023-02-12 04:00:57,519 INFO [StoreOpener-14a5b26b7821b29d8e7405e41ed6325f-1] regionserver.HStore(310): Store=14a5b26b7821b29d8e7405e41ed6325f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:57,519 INFO [StoreOpener-14a5b26b7821b29d8e7405e41ed6325f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:00:57,521 DEBUG [StoreOpener-14a5b26b7821b29d8e7405e41ed6325f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2 2023-02-12 04:00:57,521 DEBUG [StoreOpener-14a5b26b7821b29d8e7405e41ed6325f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2 2023-02-12 04:00:57,522 INFO [StoreOpener-14a5b26b7821b29d8e7405e41ed6325f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 14a5b26b7821b29d8e7405e41ed6325f columnFamilyName cf2 2023-02-12 04:00:57,524 INFO [StoreOpener-14a5b26b7821b29d8e7405e41ed6325f-1] regionserver.HStore(310): Store=14a5b26b7821b29d8e7405e41ed6325f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:00:57,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:00:57,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:00:57,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:00:57,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:00:57,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 04:00:57,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 14a5b26b7821b29d8e7405e41ed6325f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62762072, jitterRate=-0.06477224826812744}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:00:57,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 14a5b26b7821b29d8e7405e41ed6325f: Running coprocessor pre-open hook at 1676174457515Writing region info on filesystem at 1676174457515Initializing all the Stores at 1676174457516 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174457516Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174457516Cleaning up temporary data from old regions at 1676174457526 (+10 ms)Cleaning up detritus from prior splits at 1676174457526Running coprocessor post-open hooks at 1676174457533 (+7 ms)Region opened successfully at 1676174457533 2023-02-12 04:00:57,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f., pid=27, masterSystemTime=1676174457510 2023-02-12 04:00:57,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:57,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:57,537 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=14a5b26b7821b29d8e7405e41ed6325f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,537 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174457537"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174457537"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174457537"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174457537"}]},"ts":"1676174457537"} 2023-02-12 04:00:57,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-12 04:00:57,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 in 181 msec 2023-02-12 04:00:57,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-12 04:00:57,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, ASSIGN in 341 msec 2023-02-12 04:00:57,549 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 04:00:57,549 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174457549"}]},"ts":"1676174457549"} 2023-02-12 04:00:57,551 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-12 04:00:57,554 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 04:00:57,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 399 msec 2023-02-12 04:00:57,668 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:00:57,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-12 04:00:57,671 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-12 04:00:57,671 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:57,672 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-12 04:00:57,672 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-12 04:00:57,672 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,672 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,673 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:57,673 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,673 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:57,673 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:57,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:57,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:57,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:57,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:57,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:57,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:57,674 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,674 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:00:57,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,674 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,674 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,675 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:57,675 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,675 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:57,675 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:57,675 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:57,675 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:57,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,676 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:57,681 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:57,682 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:57,682 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:57,682 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:57,682 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:57,682 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:57,682 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:00:57,683 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:57,683 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:00:57,683 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:57,683 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:57,683 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:57,682 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:57,683 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. started... 2023-02-12 04:00:57,683 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:57,684 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:57,684 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:57,684 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 14a5b26b7821b29d8e7405e41ed6325f 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-12 04:00:57,686 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,686 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,686 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,686 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:57,686 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,686 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:57,700 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/a126935f7e3749e393ef686a1abcc7d7 2023-02-12 04:00:57,719 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/19d9f0fe771f4e31832b51797912c11c 2023-02-12 04:00:57,725 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/a126935f7e3749e393ef686a1abcc7d7 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7 2023-02-12 04:00:57,731 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:00:57,732 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/19d9f0fe771f4e31832b51797912c11c as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c 2023-02-12 04:00:57,738 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:00:57,739 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 14a5b26b7821b29d8e7405e41ed6325f in 54ms, sequenceid=6, compaction requested=false 2023-02-12 04:00:57,739 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-12 04:00:57,739 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 14a5b26b7821b29d8e7405e41ed6325f: 2023-02-12 04:00:57,739 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:57,739 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:00:57,739 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:00:57,739 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,739 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:57,739 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:57,740 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,740 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:57,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:57,741 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,741 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:57,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:57,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:57,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:57,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:57,743 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-12 04:00:57,743 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58b609c7[Count = 0] remaining members to acquire global barrier 2023-02-12 04:00:57,743 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-12 04:00:57,743 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,744 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,744 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,744 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,744 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:57,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,744 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,744 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:00:57,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,744 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:57,744 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:57,744 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:00:57,744 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:57,744 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:57,744 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:57,744 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:00:57,744 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:00:57,745 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,745 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:57,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:57,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:57,745 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:57,745 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:57,746 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:57,746 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:57,746 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:57,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:57,746 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:57,746 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:57,747 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:57,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:57,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:57,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:57,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:57,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:00:57,750 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-12 04:00:57,750 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-12 04:00:57,750 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:00:57,750 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:00:57,750 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-12 04:00:57,750 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:00:57,752 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,752 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,752 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:57,752 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:57,752 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,752 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:57,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:57,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,752 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:57,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,752 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:57,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:57,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:57,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:57,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:57,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:57,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:57,753 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:57,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,753 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:57,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:57,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:57,754 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:57,754 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,755 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,757 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:57,757 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:57,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:57,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-12 04:00:57,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:00:57,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:57,757 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:57,757 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:57,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:57,757 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:57,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:57,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:57,757 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-12 04:00:57,758 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:00:57,757 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,757 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:57,758 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:57,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:57,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:57,757 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:57,758 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:57,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:57,758 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,758 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,759 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:57,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:57,759 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,759 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,759 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,759 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,759 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,858 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:00:57,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:00:57,881 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:00:57,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-12 04:00:57,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-12 04:00:57,883 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-12 04:00:57,884 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:57,884 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-12 04:00:57,884 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-12 04:00:57,885 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,885 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,886 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,886 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:57,886 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:57,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:57,886 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:57,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:57,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:57,886 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,886 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:00:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:57,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:57,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:57,887 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:57,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,887 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:57,887 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:57,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:57,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:57,888 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:57,888 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:57,888 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:57,888 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:57,888 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:57,888 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:57,890 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:57,891 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:57,891 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:57,891 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:57,891 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:57,891 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:57,892 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:57,892 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:57,892 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:57,891 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:57,892 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:57,892 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:00:57,892 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:00:57,892 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:57,892 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. started... 2023-02-12 04:00:57,892 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 14a5b26b7821b29d8e7405e41ed6325f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:00:57,893 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,893 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,893 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,893 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:57,893 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,893 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:57,905 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/d2eb7892040444ba95d362965b89dfa7 2023-02-12 04:00:57,921 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/d001b707ba4e4e7189b2b81c6e20a812 2023-02-12 04:00:57,926 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/d2eb7892040444ba95d362965b89dfa7 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7 2023-02-12 04:00:57,932 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:00:57,933 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/d001b707ba4e4e7189b2b81c6e20a812 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812 2023-02-12 04:00:57,938 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:00:57,939 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 14a5b26b7821b29d8e7405e41ed6325f in 47ms, sequenceid=19, compaction requested=false 2023-02-12 04:00:57,939 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 14a5b26b7821b29d8e7405e41ed6325f: 2023-02-12 04:00:57,939 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:57,939 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:00:57,939 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:00:57,939 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,939 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:57,939 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:57,940 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,940 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:57,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:57,941 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,941 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:57,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:57,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:57,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:57,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:57,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-12 04:00:57,942 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@692d46f0[Count = 0] remaining members to acquire global barrier 2023-02-12 04:00:57,942 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-12 04:00:57,943 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,943 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,943 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,943 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,944 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,944 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:00:57,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,944 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:57,944 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:57,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,944 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:00:57,944 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:57,944 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:57,944 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:57,944 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:00:57,944 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:57,944 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:00:57,945 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:57,946 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:57,946 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:57,947 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:57,947 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,947 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:57,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:57,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:57,948 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:57,947 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:57,948 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:57,948 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:57,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:57,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:57,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:57,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:57,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:57,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:00:57,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-12 04:00:57,951 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-12 04:00:57,952 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:00:57,952 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:00:57,952 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-12 04:00:57,952 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:00:57,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:57,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:57,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:57,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,962 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:57,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:57,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,962 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:57,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:57,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,962 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:57,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:57,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:57,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,963 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:57,963 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:57,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:57,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,965 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:57,965 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:57,966 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,973 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:57,973 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:57,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:57,973 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,973 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:57,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:57,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:57,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:57,973 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:57,973 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:57,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:57,973 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-12 04:00:57,973 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:00:57,973 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:57,973 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:57,973 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:57,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:57,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:57,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,974 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-12 04:00:57,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:57,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:57,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:57,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:57,975 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:00:57,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:57,975 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:57,975 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,975 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:57,975 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:57,975 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:57,975 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,075 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:00:58,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:00:58,098 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:00:58,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-12 04:00:58,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-12 04:00:58,100 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-12 04:00:58,100 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:58,101 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-12 04:00:58,101 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-12 04:00:58,102 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,102 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,102 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,103 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,102 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,103 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,103 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,103 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:00:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,104 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:58,104 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:58,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:58,104 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:58,104 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:58,104 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:58,104 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:58,104 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:58,104 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:58,107 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:58,107 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:58,107 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:58,107 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:58,107 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:58,108 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:58,108 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:00:58,108 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:00:58,108 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:58,108 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:58,108 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:58,109 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:58,109 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. started... 2023-02-12 04:00:58,109 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:58,109 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:58,109 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 14a5b26b7821b29d8e7405e41ed6325f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:00:58,110 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,110 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,110 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,110 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:58,110 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,110 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:58,122 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87 2023-02-12 04:00:58,141 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/81605c0591ab4ce591a77144516216a7 2023-02-12 04:00:58,149 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87 2023-02-12 04:00:58,155 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:00:58,157 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/81605c0591ab4ce591a77144516216a7 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7 2023-02-12 04:00:58,163 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:00:58,165 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 14a5b26b7821b29d8e7405e41ed6325f in 55ms, sequenceid=32, compaction requested=true 2023-02-12 04:00:58,165 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 14a5b26b7821b29d8e7405e41ed6325f: 2023-02-12 04:00:58,165 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:58,165 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:00:58,165 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:00:58,165 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,165 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:58,165 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:58,166 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,166 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:58,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:58,166 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,166 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:58,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:58,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:58,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:58,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:58,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-12 04:00:58,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d2ec2d8[Count = 0] remaining members to acquire global barrier 2023-02-12 04:00:58,168 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-12 04:00:58,168 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,169 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,169 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,169 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:58,169 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:58,169 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:58,169 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,169 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:00:58,169 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:58,169 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,170 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:00:58,170 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:00:58,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,170 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:58,170 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:58,170 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:00:58,171 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:58,171 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:58,171 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:58,172 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,172 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:58,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:58,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:58,172 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:58,172 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:58,172 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:58,172 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:58,173 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:58,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:58,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:58,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:58,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:58,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:58,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:00:58,177 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-12 04:00:58,177 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-12 04:00:58,177 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:00:58,177 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:00:58,177 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-12 04:00:58,177 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:00:58,181 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,181 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,181 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:58,181 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,181 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:58,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:58,181 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:58,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:58,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:58,181 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:58,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:58,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:58,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:58,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:58,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:58,182 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:58,183 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,183 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:58,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:58,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,184 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:58,185 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:58,185 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,189 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,189 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,189 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:58,189 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,190 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,190 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,190 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,190 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,190 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,190 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,190 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,190 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,190 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,190 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:58,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,190 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-12 04:00:58,191 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:00:58,190 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,190 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,191 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:58,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,191 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:58,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,191 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-12 04:00:58,191 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:00:58,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:58,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:58,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:58,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:58,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:58,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:58,292 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:00:58,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:00:58,295 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-12 04:00:58,295 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-12 04:00:58,296 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:00:58,296 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-12 04:00:58,316 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:00:58,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-12 04:00:58,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-12 04:00:58,318 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-12 04:00:58,318 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:58,319 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-12 04:00:58,319 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-12 04:00:58,320 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,320 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,321 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,321 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,321 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,321 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,322 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,322 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:00:58,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:58,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,322 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:58,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:58,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:58,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,323 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:58,323 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:58,323 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:58,323 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:58,323 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:58,325 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:58,325 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:58,323 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:58,326 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:58,326 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:58,325 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:58,326 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:58,326 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:58,326 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:58,326 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:58,326 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:58,326 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:58,326 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:58,328 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:58,328 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:58,328 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:58,328 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:00:58,328 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:00:58,328 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:58,329 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. started... 2023-02-12 04:00:58,329 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 14a5b26b7821b29d8e7405e41ed6325f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:00:58,336 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,336 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,337 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,337 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:58,337 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,337 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:58,338 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/27935940fb9f48ad95a73bd0b7caa291 2023-02-12 04:00:58,755 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/d853e69474094f10a25af38df7dadeb5 2023-02-12 04:00:58,761 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/27935940fb9f48ad95a73bd0b7caa291 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291 2023-02-12 04:00:58,767 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:00:58,768 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/d853e69474094f10a25af38df7dadeb5 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5 2023-02-12 04:00:58,773 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:00:58,774 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 14a5b26b7821b29d8e7405e41ed6325f in 445ms, sequenceid=45, compaction requested=true 2023-02-12 04:00:58,774 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 14a5b26b7821b29d8e7405e41ed6325f: 2023-02-12 04:00:58,775 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:58,775 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:00:58,775 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:00:58,775 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,775 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:58,775 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:58,776 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,776 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:58,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:58,777 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,777 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:58,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:58,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:58,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:58,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:58,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-12 04:00:58,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@602e1181[Count = 0] remaining members to acquire global barrier 2023-02-12 04:00:58,778 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-12 04:00:58,778 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,779 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,779 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,779 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,779 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:58,779 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,779 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:00:58,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,779 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:58,779 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:58,780 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:58,779 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:00:58,779 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:58,780 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:58,780 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:00:58,780 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:00:58,780 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:58,780 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:58,780 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:58,781 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:58,781 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:58,782 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:58,781 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,782 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:58,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,782 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:58,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:58,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:58,782 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:58,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:58,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:58,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:58,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:00:58,787 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-12 04:00:58,787 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-12 04:00:58,787 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:00:58,787 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:00:58,787 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-12 04:00:58,787 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:00:58,794 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,794 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,794 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,794 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,794 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:58,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:58,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:58,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,794 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:58,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,795 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:58,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,794 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:58,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:58,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:58,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:58,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:58,795 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:58,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:58,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:58,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:58,796 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:58,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:58,797 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:58,798 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,798 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,803 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,803 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,803 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,803 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:58,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,803 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:58,803 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-12 04:00:58,803 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:00:58,803 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,803 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,803 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:58,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:58,803 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:58,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,803 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,803 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-12 04:00:58,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:58,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:58,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:58,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:58,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:58,804 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:00:58,804 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,804 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,804 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,804 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,804 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,804 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,804 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,804 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,904 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:00:58,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:00:58,928 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:00:58,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-12 04:00:58,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-12 04:00:58,932 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-12 04:00:58,932 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:58,932 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-12 04:00:58,932 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-12 04:00:58,933 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,933 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,934 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,934 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,934 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,934 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:58,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,934 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,934 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:00:58,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:58,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:58,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:58,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:58,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,935 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:58,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:58,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,935 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:58,935 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:58,936 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:58,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:58,936 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:58,936 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:58,936 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:58,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:58,936 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:58,936 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:58,936 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:58,936 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:58,936 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:58,936 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:58,936 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:58,936 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:58,936 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:58,936 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:58,937 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,937 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,937 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:58,937 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,937 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,938 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:58,939 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:58,939 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:58,939 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:58,939 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:58,939 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:58,939 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:00:58,939 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:00:58,939 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:58,939 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. started... 2023-02-12 04:00:58,939 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 14a5b26b7821b29d8e7405e41ed6325f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:00:58,955 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/9ecc84a75e1f4f0786319d6a716e7e5e 2023-02-12 04:00:58,971 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/e35dd2eab4ee421abccb2922161727b2 2023-02-12 04:00:58,978 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/9ecc84a75e1f4f0786319d6a716e7e5e as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e 2023-02-12 04:00:58,984 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:00:58,985 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/e35dd2eab4ee421abccb2922161727b2 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2 2023-02-12 04:00:58,991 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:00:58,993 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 14a5b26b7821b29d8e7405e41ed6325f in 54ms, sequenceid=58, compaction requested=true 2023-02-12 04:00:58,993 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 14a5b26b7821b29d8e7405e41ed6325f: 2023-02-12 04:00:58,993 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:58,993 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:00:58,993 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:00:58,993 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,993 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:58,993 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:58,994 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,994 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:58,995 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,995 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:58,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-12 04:00:58,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@83ed472[Count = 0] remaining members to acquire global barrier 2023-02-12 04:00:58,997 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-12 04:00:58,997 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,998 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,998 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,998 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,998 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:58,998 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:58,998 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:58,999 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:00:58,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,998 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:00:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:58,999 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:58,999 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:58,999 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:58,999 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:58,999 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:00:58,999 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:00:59,000 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:59,000 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:59,000 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:59,000 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:59,000 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:59,000 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,000 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:59,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:59,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:59,000 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:59,000 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:59,000 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:59,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:59,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:59,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:59,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:00:59,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-12 04:00:59,004 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-12 04:00:59,004 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:00:59,004 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:00:59,004 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-12 04:00:59,004 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:00:59,006 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,006 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:59,006 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,006 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,006 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:59,006 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:59,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,006 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:59,006 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:59,007 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:59,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:59,007 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:59,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:59,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,008 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,008 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:59,009 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,011 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,011 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:59,011 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,011 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:59,011 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:59,011 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:59,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:59,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:59,011 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:59,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:59,011 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,011 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:59,011 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:59,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:59,011 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-12 04:00:59,011 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:00:59,011 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:59,011 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:59,011 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:59,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:59,011 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:59,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:59,012 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-12 04:00:59,012 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:00:59,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:59,012 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:59,012 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,012 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:59,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:59,012 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,112 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:00:59,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:00:59,133 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:00:59,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-12 04:00:59,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-12 04:00:59,135 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-12 04:00:59,136 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:59,136 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-12 04:00:59,136 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-12 04:00:59,136 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,136 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:59,137 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:59,137 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:59,137 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:59,137 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:59,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:59,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:59,137 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:59,137 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:00:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-12 04:00:59,138 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:59,138 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:59,138 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:59,138 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:59,138 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:59,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-12 04:00:59,139 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:59,139 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:59,139 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:59,139 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:59,139 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-12 04:00:59,139 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:00:59,139 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:59,139 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-12 04:00:59,139 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:59,139 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:59,139 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:59,139 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. started... 2023-02-12 04:00:59,139 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:00:59,139 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:00:59,139 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:59,139 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 14a5b26b7821b29d8e7405e41ed6325f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:00:59,140 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,140 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,140 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,140 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:59,140 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,141 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:59,150 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/dbad06fd86c64e4b8afde76bb1a28052 2023-02-12 04:00:59,167 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/3255711fd2fb41868da8a0dffcbbb6c5 2023-02-12 04:00:59,173 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/dbad06fd86c64e4b8afde76bb1a28052 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052 2023-02-12 04:00:59,178 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:00:59,180 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/3255711fd2fb41868da8a0dffcbbb6c5 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5 2023-02-12 04:00:59,185 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:00:59,186 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 14a5b26b7821b29d8e7405e41ed6325f in 47ms, sequenceid=71, compaction requested=true 2023-02-12 04:00:59,186 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 14a5b26b7821b29d8e7405e41ed6325f: 2023-02-12 04:00:59,186 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:00:59,187 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:00:59,187 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:00:59,187 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,187 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-12 04:00:59,187 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-12 04:00:59,188 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,188 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:59,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:59,188 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,188 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:00:59,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:59,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:59,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:59,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:59,190 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-12 04:00:59,190 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ba71943[Count = 0] remaining members to acquire global barrier 2023-02-12 04:00:59,190 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-12 04:00:59,190 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,191 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,191 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,191 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,191 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,191 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:00:59,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,191 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:59,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,192 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:59,192 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:59,192 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:00:59,192 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-12 04:00:59,192 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:59,192 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-12 04:00:59,192 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:00:59,192 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:00:59,192 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:59,193 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:59,193 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,193 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:59,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:59,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:59,193 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:59,193 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-12 04:00:59,193 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:59,193 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:59,193 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:59,193 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-12 04:00:59,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:59,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:59,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:59,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:59,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:59,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:00:59,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-12 04:00:59,196 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-12 04:00:59,196 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:00:59,196 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:00:59,196 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-12 04:00:59,196 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:00:59,197 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,197 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:59,197 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,197 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,197 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:59,197 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,197 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:59,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:59,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:00:59,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:59,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:59,197 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:59,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:00:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,198 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:00:59,198 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:59,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:00:59,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:59,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,200 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:00:59,200 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-12 04:00:59,200 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:59,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:59,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:59,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:59,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:59,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:00:59,202 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-12 04:00:59,202 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:00:59,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:59,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:00:59,202 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:00:59,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:59,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:00:59,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:59,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,202 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-12 04:00:59,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:59,202 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:00:59,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-12 04:00:59,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:00:59,203 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:00:59,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:59,203 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:59,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:59,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:00:59,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:59,203 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:00:59,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:00:59,203 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,203 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-12 04:00:59,203 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-12 04:00:59,303 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:00:59,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:00:59,569 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-12 04:00:59,570 DEBUG [Listener at localhost.localdomain/35207] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:00:59,571 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:00:59,572 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:00:59,572 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:00:59,573 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:00:59,574 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:00:59,574 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:00:59,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:00:59,574 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:00:59,822 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:00:59,823 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:00:59,823 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:00:59,823 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:00:59,825 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-12 04:00:59,825 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:00:59,825 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:00:59,825 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:00:59,826 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:00:59,826 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:00:59,830 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-12 04:00:59,831 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:00:59,831 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:00:59,832 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:00:59,832 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:00:59,832 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:00:59,832 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:00:59,833 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:00:59,833 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:00:59,833 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:00:59,834 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:00:59,834 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:00:59,834 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:00:59,834 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:00:59,834 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:00:59,835 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-12 04:00:59,835 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:00:59,836 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:00:59,837 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174489837 2023-02-12 04:00:59,838 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:00:59,838 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:00:59,842 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:00:59,842 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=14a5b26b7821b29d8e7405e41ed6325f, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:00:59,842 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:00:59,842 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174519842 2023-02-12 04:00:59,843 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174579843 2023-02-12 04:00:59,843 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-12 04:00:59,843 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:00:59,845 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:00:59,845 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:00:59,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:00:59,845 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:00,093 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,093 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:00,093 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:00,093 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:00,093 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:00,094 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:00,094 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:00,094 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:00,094 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:00,095 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:00,096 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-12 04:01:00,097 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:00,097 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:00,097 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:00,098 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:00,098 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:00,098 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:00,098 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:00,099 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:00,099 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:00,099 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,100 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:00,100 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,100 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,100 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,100 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-12 04:01:00,101 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:00,101 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:00,102 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174490102 2023-02-12 04:01:00,103 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:00,103 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,106 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:00,106 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=14a5b26b7821b29d8e7405e41ed6325f, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:00,106 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:00,106 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174520106 2023-02-12 04:01:00,106 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174580106 2023-02-12 04:01:00,106 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:00,106 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:00,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN}] 2023-02-12 04:01:00,111 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:00,111 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:00,111 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:00,356 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,357 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:00,357 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:00,357 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:00,357 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:00,357 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:00,357 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:00,357 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:00,357 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:00,358 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:00,359 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-12 04:01:00,360 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:00,360 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:00,360 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:00,361 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:00,361 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:00,362 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:00,362 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:00,362 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:00,362 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:00,363 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,363 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:00,363 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,363 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,363 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,363 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-12 04:01:00,364 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:00,364 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:00,368 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174490368 2023-02-12 04:01:00,369 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,369 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:00,372 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:00,372 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=14a5b26b7821b29d8e7405e41ed6325f, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:00,372 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:00,373 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174520373 2023-02-12 04:01:00,373 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174580373 2023-02-12 04:01:00,373 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:00,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN}] 2023-02-12 04:01:00,373 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:00,376 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:00,377 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=14a5b26b7821b29d8e7405e41ed6325f, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:00,377 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174460377"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174460377"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174460377"}]},"ts":"1676174460377"} 2023-02-12 04:01:00,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:01:00,379 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:00,379 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:00,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:00,379 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:00,623 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,623 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:00,623 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:00,623 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:00,623 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:00,623 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:00,624 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:00,624 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:00,624 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:00,624 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:00,626 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-12 04:01:00,626 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-12 04:01:00,626 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:00,627 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:00,627 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:00,627 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:00,627 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:00,628 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:00,628 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:00,628 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:00,628 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:00,629 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,629 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:00,629 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:00,629 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,629 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,630 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,630 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:00,630 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:00,630 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-12 04:01:00,631 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:00,631 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:00,631 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f to restore RIT 2023-02-12 04:01:00,631 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174490631 2023-02-12 04:01:00,632 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:00,632 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:00,635 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:00,635 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=14a5b26b7821b29d8e7405e41ed6325f, regionState=CLOSING, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:00,635 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:00,636 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174520635 2023-02-12 04:01:00,636 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174580636 2023-02-12 04:01:00,636 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:00,636 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=14a5b26b7821b29d8e7405e41ed6325f, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:00,636 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174460636"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174460636"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174460636"}]},"ts":"1676174460636"} 2023-02-12 04:01:00,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:01:00,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:00,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:00,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 14a5b26b7821b29d8e7405e41ed6325f, disabling compactions & flushes 2023-02-12 04:01:00,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:01:00,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:01:00,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. after waiting 0 ms 2023-02-12 04:01:00,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:01:00,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 14a5b26b7821b29d8e7405e41ed6325f 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-12 04:01:00,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f 2023-02-12 04:01:00,822 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/8d6d0d0405424b6bb655b3383e05c4f8 2023-02-12 04:01:00,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f 2023-02-12 04:01:00,835 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:01:00,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/.tmp/cf2/8d6d0d0405424b6bb655b3383e05c4f8 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8 2023-02-12 04:01:00,842 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:01:00,843 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 14a5b26b7821b29d8e7405e41ed6325f in 51ms, sequenceid=82, compaction requested=true 2023-02-12 04:01:00,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-12 04:01:00,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:01:00,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. 2023-02-12 04:01:00,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 14a5b26b7821b29d8e7405e41ed6325f: Waiting for close lock at 1676174460792Running coprocessor pre-close hooks at 1676174460792Disabling compacts and flushes for region at 1676174460792Disabling writes for close at 1676174460792Obtaining lock to block concurrent updates at 1676174460792Preparing flush snapshotting stores in 14a5b26b7821b29d8e7405e41ed6325f at 1676174460792Finished memstore snapshotting testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676174460793 (+1 ms)Flushing stores of testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. at 1676174460793Flushing 14a5b26b7821b29d8e7405e41ed6325f/cf1: creating writer at 1676174460794 (+1 ms)Flushing 14a5b26b7821b29d8e7405e41ed6325f/cf1: appending metadata at 1676174460796 (+2 ms)Flushing 14a5b26b7821b29d8e7405e41ed6325f/cf1: closing flushed file at 1676174460796Flushing 14a5b26b7821b29d8e7405e41ed6325f/cf2: creating writer at 1676174460809 (+13 ms)Flushing 14a5b26b7821b29d8e7405e41ed6325f/cf2: appending metadata at 1676174460811 (+2 ms)Flushing 14a5b26b7821b29d8e7405e41ed6325f/cf2: closing flushed file at 1676174460811Flushing 14a5b26b7821b29d8e7405e41ed6325f/cf1: reopening flushed file at 1676174460829 (+18 ms)Flushing 14a5b26b7821b29d8e7405e41ed6325f/cf2: reopening flushed file at 1676174460836 (+7 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 14a5b26b7821b29d8e7405e41ed6325f in 51ms, sequenceid=82, compaction requested=true at 1676174460843 (+7 ms)Writing region close event to WAL at 1676174460862 (+19 ms)Running coprocessor post-close hooks at 1676174460866 (+4 ms)Closed at 1676174460867 (+1 ms) 2023-02-12 04:01:00,870 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:00,870 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=14a5b26b7821b29d8e7405e41ed6325f, regionState=CLOSED 2023-02-12 04:01:00,870 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174460870"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174460870"}]},"ts":"1676174460870"} 2023-02-12 04:01:00,872 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:00,872 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:00,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:00,872 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:00,886 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,886 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:00,886 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:00,886 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:00,886 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:00,887 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:00,887 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:00,887 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:00,887 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:00,887 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:00,889 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-12 04:01:00,889 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-12 04:01:00,889 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-12 04:01:00,889 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:00,890 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:00,890 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:00,890 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:00,890 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:00,890 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:00,891 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:00,891 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:00,891 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:00,892 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,892 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:00,892 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:00,892 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:00,892 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,892 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,892 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:00,892 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:00,892 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:00,892 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:00,892 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-12 04:01:00,893 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:00,894 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:00,894 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f to restore RIT 2023-02-12 04:01:00,894 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174490894 2023-02-12 04:01:00,894 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:00,895 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:00,897 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:00,897 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=14a5b26b7821b29d8e7405e41ed6325f, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:00,898 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:00,898 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174520898 2023-02-12 04:01:00,898 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174580898 2023-02-12 04:01:00,898 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:00,898 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=14a5b26b7821b29d8e7405e41ed6325f, regionState=CLOSED, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:00,898 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174460898"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174460898"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174460898"}]},"ts":"1676174460898"} 2023-02-12 04:01:00,900 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:00,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-12 04:01:00,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 in 262 msec 2023-02-12 04:01:00,904 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:00,904 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:00,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:00,904 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:01,148 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,148 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:01,148 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:01,148 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:01,149 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:01,149 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:01,149 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:01,149 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:01,149 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:01,150 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:01,151 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-12 04:01:01,151 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-12 04:01:01,151 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:01,151 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:01,152 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:01,152 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:01,152 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:01,153 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:01,153 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:01,153 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:01,153 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:01,154 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:01,154 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,154 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:01,154 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:01,154 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:01,155 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,155 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,155 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,155 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:01,155 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:01,155 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-12 04:01:01,156 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:01,156 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:01,156 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f to restore RIT 2023-02-12 04:01:01,157 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174491157 2023-02-12 04:01:01,157 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:01,158 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:01,160 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:01,160 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=14a5b26b7821b29d8e7405e41ed6325f, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:01,160 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:01,161 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174521161 2023-02-12 04:01:01,161 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174581161 2023-02-12 04:01:01,161 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:01,161 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:01,163 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:01,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-12 04:01:01,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN in 790 msec 2023-02-12 04:01:01,169 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:01,169 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:01,169 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:01,411 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,411 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:01,411 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:01,411 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:01,411 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:01,412 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:01,412 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:01,412 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:01,412 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:01,413 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:01,414 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-12 04:01:01,414 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:01:01,414 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:01,414 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:01,414 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:01,415 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:01,415 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:01,415 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:01,415 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:01,415 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:01,416 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:01,416 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:01,416 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,417 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:01,417 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:01,417 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:01,417 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:01,417 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:01,417 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-12 04:01:01,418 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:01,418 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:01,418 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174491418 2023-02-12 04:01:01,420 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,420 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:01,422 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:01,423 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=14a5b26b7821b29d8e7405e41ed6325f, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:01,423 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:01,423 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174521423 2023-02-12 04:01:01,423 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174581423 2023-02-12 04:01:01,423 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:01,423 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,423 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,423 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,429 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:01,439 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=14a5b26b7821b29d8e7405e41ed6325f, threads=14 2023-02-12 04:01:01,440 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,440 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,442 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,444 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,445 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,445 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,445 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,445 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,445 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,445 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,447 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,447 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,447 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,448 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,615 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,635 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,647 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,647 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,655 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,655 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,663 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,669 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,670 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,672 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,675 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,675 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,677 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,678 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,678 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 14a5b26b7821b29d8e7405e41ed6325f Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-12 04:01:01,719 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:01,719 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:01,720 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:01,923 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,923 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:01,924 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:01,924 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:01,924 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:01,924 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:01,924 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:01,924 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:01,924 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:01,925 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:01,926 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-12 04:01:01,926 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:01:01,926 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:01,927 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:01,927 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:01,927 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:01,928 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:01,928 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:01,928 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:01,929 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:01,929 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:01,929 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:01,930 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,930 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 04:01:01,930 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:01,930 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:01,930 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:01,931 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,931 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,931 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,931 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:01,931 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:01,931 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-12 04:01:01,932 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:01,932 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:01,933 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174491933 2023-02-12 04:01:01,933 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:01,935 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:01,937 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:01,937 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=14a5b26b7821b29d8e7405e41ed6325f, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:01,937 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:01,937 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174521937 2023-02-12 04:01:01,937 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174581937 2023-02-12 04:01:01,937 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-12 04:01:01,949 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=14a5b26b7821b29d8e7405e41ed6325f, threads=14 2023-02-12 04:01:01,951 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,951 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,951 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,954 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,954 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,954 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,954 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,954 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,955 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,960 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,961 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,963 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,971 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:01,975 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,194 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,202 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,215 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,215 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,229 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,229 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,235 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,248 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,259 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,267 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,268 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,271 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,274 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,275 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812 for region: 14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:02,275 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 14a5b26b7821b29d8e7405e41ed6325f Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-12 04:01:02,317 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:02,323 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-12 04:01:02,325 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-12 04:01:02,325 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:02,325 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:02,325 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:02,325 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:02,438 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:02,438 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:02,438 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:02,438 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:02,438 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:02,438 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:02,438 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:02,438 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:02,438 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:02,439 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:02,440 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-12 04:01:02,440 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:01:02,440 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:02,441 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:02,441 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:02,441 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:02,441 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:02,442 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:02,442 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:02,442 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:02,442 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:02,442 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:02,443 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:02,443 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:02,443 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:02,443 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:02,443 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:02,443 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:02,443 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:02,443 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:02,443 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:02,444 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-12 04:01:02,444 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:02,445 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:02,445 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174492445 2023-02-12 04:01:02,447 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:02,447 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:02,449 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:02,449 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=14a5b26b7821b29d8e7405e41ed6325f, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:02,449 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:02,449 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174522449 2023-02-12 04:01:02,449 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174582449 2023-02-12 04:01:02,449 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-12 04:01:02,452 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:02,455 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:02,455 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:02,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:02,455 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:02,700 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:01:02,700 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:02,700 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:02,700 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:02,701 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:02,701 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:02,701 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:02,701 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:02,701 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:02,702 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:02,703 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-12 04:01:02,703 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:01:02,703 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:02,703 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:02,703 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:02,703 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:02,704 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:02,704 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:02,704 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:02,704 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:02,704 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:02,705 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:02,705 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:02,705 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN 2023-02-12 04:01:02,706 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:02,706 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:02,706 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:02,706 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:02,706 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:02,706 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=14a5b26b7821b29d8e7405e41ed6325f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:02,706 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 14a5b26b7821b29d8e7405e41ed6325f, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:02,706 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-12 04:01:02,708 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:02,708 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:02,709 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174492709 2023-02-12 04:01:02,710 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:02,710 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:02,712 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:02,712 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=14a5b26b7821b29d8e7405e41ed6325f, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:02,713 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:02,713 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174522713 2023-02-12 04:01:02,713 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174582713 2023-02-12 04:01:02,713 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:02,716 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174462715"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676174462715"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676174462715"}]},"ts":"1676174462715"} 2023-02-12 04:01:02,716 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174462715"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174462715"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174462715"}]},"ts":"1676174462715"} 2023-02-12 04:01:02,716 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174462715"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174462715"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174462715"}]},"ts":"1676174462715"} 2023-02-12 04:01:02,722 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2405ca844e0b3394a8e057a0063d11f6, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3d0c2ff996cdb076c77801003013a683, ASSIGN}] 2023-02-12 04:01:02,723 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3d0c2ff996cdb076c77801003013a683, ASSIGN 2023-02-12 04:01:02,724 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3d0c2ff996cdb076c77801003013a683, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,43977,1676174449184; forceNewPlan=false, retain=false 2023-02-12 04:01:02,725 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2405ca844e0b3394a8e057a0063d11f6, ASSIGN 2023-02-12 04:01:02,726 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2405ca844e0b3394a8e057a0063d11f6, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,43977,1676174449184; forceNewPlan=false, retain=false 2023-02-12 04:01:02,875 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-12 04:01:02,876 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=3d0c2ff996cdb076c77801003013a683, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:02,876 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174462876"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174462876"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174462876"}]},"ts":"1676174462876"} 2023-02-12 04:01:02,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 3d0c2ff996cdb076c77801003013a683, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:01:02,881 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=2405ca844e0b3394a8e057a0063d11f6, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:02,882 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174462881"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174462881"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174462881"}]},"ts":"1676174462881"} 2023-02-12 04:01:02,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure 2405ca844e0b3394a8e057a0063d11f6, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:01:03,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683. 2023-02-12 04:01:03,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3d0c2ff996cdb076c77801003013a683, NAME => 'testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.', STARTKEY => '41', ENDKEY => ''} 2023-02-12 04:01:03,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:01:03,035 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:03,035 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:03,035 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:03,035 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:03,036 INFO [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:03,037 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1 2023-02-12 04:01:03,037 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1 2023-02-12 04:01:03,037 INFO [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3d0c2ff996cdb076c77801003013a683 columnFamilyName cf1 2023-02-12 04:01:03,054 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291-top 2023-02-12 04:01:03,059 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e-top 2023-02-12 04:01:03,064 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7-top 2023-02-12 04:01:03,069 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87-top 2023-02-12 04:01:03,074 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7-top 2023-02-12 04:01:03,078 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052-top 2023-02-12 04:01:03,082 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f-top 2023-02-12 04:01:03,083 INFO [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(310): Store=3d0c2ff996cdb076c77801003013a683/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:03,083 INFO [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:03,084 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2 2023-02-12 04:01:03,084 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2 2023-02-12 04:01:03,084 INFO [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3d0c2ff996cdb076c77801003013a683 columnFamilyName cf2 2023-02-12 04:01:03,100 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c-top 2023-02-12 04:01:03,106 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5-top 2023-02-12 04:01:03,111 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7-top 2023-02-12 04:01:03,115 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8-top 2023-02-12 04:01:03,120 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812-top 2023-02-12 04:01:03,125 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5-top 2023-02-12 04:01:03,131 DEBUG [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2-top 2023-02-12 04:01:03,131 INFO [StoreOpener-3d0c2ff996cdb076c77801003013a683-1] regionserver.HStore(310): Store=3d0c2ff996cdb076c77801003013a683/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:03,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:03,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:03,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:01:03,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:03,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 3d0c2ff996cdb076c77801003013a683; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66572272, jitterRate=-0.007995843887329102}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:01:03,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 3d0c2ff996cdb076c77801003013a683: Running coprocessor pre-open hook at 1676174463035Writing region info on filesystem at 1676174463035Initializing all the Stores at 1676174463036 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174463036Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174463036Cleaning up temporary data from old regions at 1676174463134 (+98 ms)Cleaning up detritus from prior splits at 1676174463134Running coprocessor post-open hooks at 1676174463137 (+3 ms)Region opened successfully at 1676174463137 2023-02-12 04:01:03,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683., pid=33, masterSystemTime=1676174463030 2023-02-12 04:01:03,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:03,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:01:03,139 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:01:03,139 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:01:03,139 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683. 2023-02-12 04:01:03,139 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683. 2023-02-12 04:01:03,139 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1912): 3d0c2ff996cdb076c77801003013a683/cf1 is initiating minor compaction (all files) 2023-02-12 04:01:03,139 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1912): 3d0c2ff996cdb076c77801003013a683/cf2 is initiating minor compaction (all files) 2023-02-12 04:01:03,139 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 3d0c2ff996cdb076c77801003013a683/cf1 in testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683. 2023-02-12 04:01:03,139 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 3d0c2ff996cdb076c77801003013a683/cf2 in testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683. 2023-02-12 04:01:03,140 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f-top] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/.tmp, totalSize=34.9 K 2023-02-12 04:01:03,140 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8-top] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/.tmp, totalSize=34.9 K 2023-02-12 04:01:03,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683. 2023-02-12 04:01:03,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683. 2023-02-12 04:01:03,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6. 2023-02-12 04:01:03,141 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676174457660 2023-02-12 04:01:03,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2405ca844e0b3394a8e057a0063d11f6, NAME => 'testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.', STARTKEY => '', ENDKEY => '41'} 2023-02-12 04:01:03,141 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676174457660 2023-02-12 04:01:03,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:01:03,142 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676174457860 2023-02-12 04:01:03,142 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=3d0c2ff996cdb076c77801003013a683, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:03,142 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676174457860 2023-02-12 04:01:03,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 2405ca844e0b3394a8e057a0063d11f6 2023-02-12 04:01:03,142 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174463142"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174463142"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174463142"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174463142"}]},"ts":"1676174463142"} 2023-02-12 04:01:03,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:03,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 2405ca844e0b3394a8e057a0063d11f6 2023-02-12 04:01:03,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 2405ca844e0b3394a8e057a0063d11f6 2023-02-12 04:01:03,142 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676174458077 2023-02-12 04:01:03,142 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676174458077 2023-02-12 04:01:03,143 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676174458294 2023-02-12 04:01:03,143 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676174458294 2023-02-12 04:01:03,143 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676174458907 2023-02-12 04:01:03,143 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676174458907 2023-02-12 04:01:03,143 INFO [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2405ca844e0b3394a8e057a0063d11f6 2023-02-12 04:01:03,144 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676174459114 2023-02-12 04:01:03,144 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676174459114 2023-02-12 04:01:03,144 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676174459305 2023-02-12 04:01:03,145 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676174459305 2023-02-12 04:01:03,147 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-02-12 04:01:03,147 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 3d0c2ff996cdb076c77801003013a683, server=jenkins-hbase20.apache.org,43977,1676174449184 in 266 msec 2023-02-12 04:01:03,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3d0c2ff996cdb076c77801003013a683, ASSIGN in 427 msec 2023-02-12 04:01:03,159 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1 2023-02-12 04:01:03,159 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1 2023-02-12 04:01:03,159 INFO [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2405ca844e0b3394a8e057a0063d11f6 columnFamilyName cf1 2023-02-12 04:01:03,162 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3d0c2ff996cdb076c77801003013a683#cf1#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:03,164 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] throttle.PressureAwareThroughputController(145): 3d0c2ff996cdb076c77801003013a683#cf2#compaction#33 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:03,181 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291-bottom 2023-02-12 04:01:03,187 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e-bottom 2023-02-12 04:01:03,193 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7-bottom 2023-02-12 04:01:03,198 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87-bottom 2023-02-12 04:01:03,205 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7-bottom 2023-02-12 04:01:03,208 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/.tmp/cf1/97027828354d43d9b3dbceaaaa5f21e2 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/97027828354d43d9b3dbceaaaa5f21e2 2023-02-12 04:01:03,211 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/.tmp/cf2/860ee2da5a80464b8b3d35e6367ac4f3 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/860ee2da5a80464b8b3d35e6367ac4f3 2023-02-12 04:01:03,211 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052-bottom 2023-02-12 04:01:03,215 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f-bottom 2023-02-12 04:01:03,215 INFO [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(310): Store=2405ca844e0b3394a8e057a0063d11f6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:03,215 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3d0c2ff996cdb076c77801003013a683/cf1 of 3d0c2ff996cdb076c77801003013a683 into 97027828354d43d9b3dbceaaaa5f21e2(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:03,216 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3d0c2ff996cdb076c77801003013a683: 2023-02-12 04:01:03,216 INFO [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2405ca844e0b3394a8e057a0063d11f6 2023-02-12 04:01:03,216 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683., storeName=3d0c2ff996cdb076c77801003013a683/cf1, priority=9, startTime=1676174463138; duration=0sec 2023-02-12 04:01:03,216 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:03,217 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2 2023-02-12 04:01:03,217 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2 2023-02-12 04:01:03,217 INFO [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2405ca844e0b3394a8e057a0063d11f6 columnFamilyName cf2 2023-02-12 04:01:03,218 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3d0c2ff996cdb076c77801003013a683/cf2 of 3d0c2ff996cdb076c77801003013a683 into 860ee2da5a80464b8b3d35e6367ac4f3(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:03,218 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3d0c2ff996cdb076c77801003013a683: 2023-02-12 04:01:03,218 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683., storeName=3d0c2ff996cdb076c77801003013a683/cf2, priority=9, startTime=1676174463139; duration=0sec 2023-02-12 04:01:03,218 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:03,234 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c-bottom 2023-02-12 04:01:03,239 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5-bottom 2023-02-12 04:01:03,244 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7-bottom 2023-02-12 04:01:03,248 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8-bottom 2023-02-12 04:01:03,252 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812-bottom 2023-02-12 04:01:03,257 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5-bottom 2023-02-12 04:01:03,265 DEBUG [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2-bottom 2023-02-12 04:01:03,265 INFO [StoreOpener-2405ca844e0b3394a8e057a0063d11f6-1] regionserver.HStore(310): Store=2405ca844e0b3394a8e057a0063d11f6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:03,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6 2023-02-12 04:01:03,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6 2023-02-12 04:01:03,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:01:03,270 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 2405ca844e0b3394a8e057a0063d11f6 2023-02-12 04:01:03,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 2405ca844e0b3394a8e057a0063d11f6; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64989752, jitterRate=-0.031577229499816895}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:01:03,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 2405ca844e0b3394a8e057a0063d11f6: Running coprocessor pre-open hook at 1676174463142Writing region info on filesystem at 1676174463142Initializing all the Stores at 1676174463143 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174463143Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174463143Cleaning up temporary data from old regions at 1676174463268 (+125 ms)Cleaning up detritus from prior splits at 1676174463268Running coprocessor post-open hooks at 1676174463271 (+3 ms)Region opened successfully at 1676174463271 2023-02-12 04:01:03,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6., pid=34, masterSystemTime=1676174463030 2023-02-12 04:01:03,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:03,273 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:01:03,273 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:01:03,273 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:01:03,273 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6. 2023-02-12 04:01:03,273 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1912): 2405ca844e0b3394a8e057a0063d11f6/cf2 is initiating minor compaction (all files) 2023-02-12 04:01:03,273 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6. 2023-02-12 04:01:03,273 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 2405ca844e0b3394a8e057a0063d11f6/cf2 in testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6. 2023-02-12 04:01:03,273 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1912): 2405ca844e0b3394a8e057a0063d11f6/cf1 is initiating minor compaction (all files) 2023-02-12 04:01:03,273 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 2405ca844e0b3394a8e057a0063d11f6/cf1 in testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6. 2023-02-12 04:01:03,274 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8-bottom] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/.tmp, totalSize=34.9 K 2023-02-12 04:01:03,274 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f-bottom] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/.tmp, totalSize=34.9 K 2023-02-12 04:01:03,274 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174457660 2023-02-12 04:01:03,275 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174457660 2023-02-12 04:01:03,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6. 2023-02-12 04:01:03,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6. 2023-02-12 04:01:03,275 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174457860 2023-02-12 04:01:03,275 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174457860 2023-02-12 04:01:03,276 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174458077 2023-02-12 04:01:03,276 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=2405ca844e0b3394a8e057a0063d11f6, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:03,276 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174463276"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174463276"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174463276"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174463276"}]},"ts":"1676174463276"} 2023-02-12 04:01:03,276 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174458294 2023-02-12 04:01:03,276 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174458077 2023-02-12 04:01:03,277 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174458907 2023-02-12 04:01:03,277 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174458294 2023-02-12 04:01:03,277 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174459114 2023-02-12 04:01:03,278 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174458907 2023-02-12 04:01:03,278 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676174459305 2023-02-12 04:01:03,279 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174459114 2023-02-12 04:01:03,279 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676174459305 2023-02-12 04:01:03,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-02-12 04:01:03,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure 2405ca844e0b3394a8e057a0063d11f6, server=jenkins-hbase20.apache.org,43977,1676174449184 in 395 msec 2023-02-12 04:01:03,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-12 04:01:03,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2405ca844e0b3394a8e057a0063d11f6, ASSIGN in 562 msec 2023-02-12 04:01:03,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 in 3.7170 sec 2023-02-12 04:01:03,297 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] throttle.PressureAwareThroughputController(145): 2405ca844e0b3394a8e057a0063d11f6#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:03,300 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2405ca844e0b3394a8e057a0063d11f6#cf1#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:03,321 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/.tmp/cf2/25018ef341fa4cca9ca95941e679b665 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/25018ef341fa4cca9ca95941e679b665 2023-02-12 04:01:03,328 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2405ca844e0b3394a8e057a0063d11f6/cf2 of 2405ca844e0b3394a8e057a0063d11f6 into 25018ef341fa4cca9ca95941e679b665(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:03,328 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2405ca844e0b3394a8e057a0063d11f6: 2023-02-12 04:01:03,328 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6., storeName=2405ca844e0b3394a8e057a0063d11f6/cf2, priority=9, startTime=1676174463273; duration=0sec 2023-02-12 04:01:03,328 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:03,329 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/.tmp/cf1/0d41704b07944ca2acbe5c6a77de00b2 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/0d41704b07944ca2acbe5c6a77de00b2 2023-02-12 04:01:03,335 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2405ca844e0b3394a8e057a0063d11f6/cf1 of 2405ca844e0b3394a8e057a0063d11f6 into 0d41704b07944ca2acbe5c6a77de00b2(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:03,335 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2405ca844e0b3394a8e057a0063d11f6: 2023-02-12 04:01:03,335 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6., storeName=2405ca844e0b3394a8e057a0063d11f6/cf1, priority=9, startTime=1676174463272; duration=0sec 2023-02-12 04:01:03,335 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:03,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43977] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 148.251.75.209:52638 deadline: 1676174523464, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f. is not online on jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:03,577 DEBUG [Listener at localhost.localdomain/35207] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-12 04:01:03,588 INFO [Listener at localhost.localdomain/35207] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-12 04:01:03,591 DEBUG [Listener at localhost.localdomain/35207] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-12 04:01:03,614 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:01:03,616 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-12 04:01:03,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testSplitWithoutPONR 2023-02-12 04:01:03,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:03,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-12 04:01:03,620 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174463620"}]},"ts":"1676174463620"} 2023-02-12 04:01:03,622 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-12 04:01:03,623 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-12 04:01:03,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2405ca844e0b3394a8e057a0063d11f6, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3d0c2ff996cdb076c77801003013a683, UNASSIGN}] 2023-02-12 04:01:03,625 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3d0c2ff996cdb076c77801003013a683, UNASSIGN 2023-02-12 04:01:03,626 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=3d0c2ff996cdb076c77801003013a683, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:03,626 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174463626"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174463626"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174463626"}]},"ts":"1676174463626"} 2023-02-12 04:01:03,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 3d0c2ff996cdb076c77801003013a683, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:01:03,630 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2405ca844e0b3394a8e057a0063d11f6, UNASSIGN 2023-02-12 04:01:03,631 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=2405ca844e0b3394a8e057a0063d11f6, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:03,631 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174463631"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174463631"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174463631"}]},"ts":"1676174463631"} 2023-02-12 04:01:03,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 2405ca844e0b3394a8e057a0063d11f6, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:01:03,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-12 04:01:03,780 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:03,780 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 3d0c2ff996cdb076c77801003013a683, disabling compactions & flushes 2023-02-12 04:01:03,781 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683. 2023-02-12 04:01:03,781 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683. 2023-02-12 04:01:03,781 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683. after waiting 0 ms 2023-02-12 04:01:03,781 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683. 2023-02-12 04:01:03,791 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f-top] to archive 2023-02-12 04:01:03,792 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:01:03,793 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,794 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,796 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,797 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,798 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,799 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,800 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,802 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8-top] to archive 2023-02-12 04:01:03,802 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:01:03,804 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,805 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,806 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,807 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,809 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,810 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,811 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-12 04:01:03,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:01:03,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683. 2023-02-12 04:01:03,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 3d0c2ff996cdb076c77801003013a683: Waiting for close lock at 1676174463780Running coprocessor pre-close hooks at 1676174463780Disabling compacts and flushes for region at 1676174463780Disabling writes for close at 1676174463781 (+1 ms)Writing region close event to WAL at 1676174463812 (+31 ms)Running coprocessor post-close hooks at 1676174463815 (+3 ms)Closed at 1676174463815 2023-02-12 04:01:03,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:03,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 2405ca844e0b3394a8e057a0063d11f6 2023-02-12 04:01:03,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 2405ca844e0b3394a8e057a0063d11f6, disabling compactions & flushes 2023-02-12 04:01:03,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6. 2023-02-12 04:01:03,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6. 2023-02-12 04:01:03,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6. after waiting 0 ms 2023-02-12 04:01:03,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6. 2023-02-12 04:01:03,817 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=3d0c2ff996cdb076c77801003013a683, regionState=CLOSED 2023-02-12 04:01:03,817 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174463817"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174463817"}]},"ts":"1676174463817"} 2023-02-12 04:01:03,818 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f-bottom] to archive 2023-02-12 04:01:03,819 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:01:03,821 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,821 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-12 04:01:03,821 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 3d0c2ff996cdb076c77801003013a683, server=jenkins-hbase20.apache.org,43977,1676174449184 in 191 msec 2023-02-12 04:01:03,823 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3d0c2ff996cdb076c77801003013a683, UNASSIGN in 197 msec 2023-02-12 04:01:03,824 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,825 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,826 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,827 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,829 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,830 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8-bottom] to archive 2023-02-12 04:01:03,831 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:01:03,833 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,835 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,836 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,837 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,839 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,841 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,842 DEBUG [StoreCloser-testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,846 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-12 04:01:03,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:01:03,847 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6. 2023-02-12 04:01:03,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 2405ca844e0b3394a8e057a0063d11f6: Waiting for close lock at 1676174463817Running coprocessor pre-close hooks at 1676174463817Disabling compacts and flushes for region at 1676174463817Disabling writes for close at 1676174463817Writing region close event to WAL at 1676174463843 (+26 ms)Running coprocessor post-close hooks at 1676174463847 (+4 ms)Closed at 1676174463847 2023-02-12 04:01:03,849 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 2405ca844e0b3394a8e057a0063d11f6 2023-02-12 04:01:03,849 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=2405ca844e0b3394a8e057a0063d11f6, regionState=CLOSED 2023-02-12 04:01:03,850 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174463849"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174463849"}]},"ts":"1676174463849"} 2023-02-12 04:01:03,853 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-12 04:01:03,853 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 2405ca844e0b3394a8e057a0063d11f6, server=jenkins-hbase20.apache.org,43977,1676174449184 in 218 msec 2023-02-12 04:01:03,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-12 04:01:03,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2405ca844e0b3394a8e057a0063d11f6, UNASSIGN in 229 msec 2023-02-12 04:01:03,857 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174463857"}]},"ts":"1676174463857"} 2023-02-12 04:01:03,858 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-12 04:01:03,859 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-12 04:01:03,861 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 244 msec 2023-02-12 04:01:03,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-12 04:01:03,923 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-12 04:01:03,923 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testSplitWithoutPONR 2023-02-12 04:01:03,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:03,926 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:03,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-12 04:01:03,927 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:03,933 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,933 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6 2023-02-12 04:01:03,933 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:03,935 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/recovered.edits] 2023-02-12 04:01:03,935 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/recovered.edits] 2023-02-12 04:01:03,935 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/recovered.edits] 2023-02-12 04:01:03,938 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/.regioninfo to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/.regioninfo 2023-02-12 04:01:03,938 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/.regioninfo to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/.regioninfo 2023-02-12 04:01:03,939 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/27935940fb9f48ad95a73bd0b7caa291 2023-02-12 04:01:03,940 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/9ecc84a75e1f4f0786319d6a716e7e5e 2023-02-12 04:01:03,941 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,942 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/a126935f7e3749e393ef686a1abcc7d7 2023-02-12 04:01:03,942 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/27935940fb9f48ad95a73bd0b7caa291.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,943 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87 2023-02-12 04:01:03,943 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,943 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/9ecc84a75e1f4f0786319d6a716e7e5e.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,944 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/d2eb7892040444ba95d362965b89dfa7 2023-02-12 04:01:03,944 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,944 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/a126935f7e3749e393ef686a1abcc7d7.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,946 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/dbad06fd86c64e4b8afde76bb1a28052 2023-02-12 04:01:03,946 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,946 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/ac3b5ef6ed264bf0a39dc17f6a13af87.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,947 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f 2023-02-12 04:01:03,947 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,947 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/d2eb7892040444ba95d362965b89dfa7.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,949 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,949 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/dbad06fd86c64e4b8afde76bb1a28052.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,951 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,951 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf1/fc5e2730bd7f4ea78c0d5f239c9eaf2f.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,951 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/19d9f0fe771f4e31832b51797912c11c 2023-02-12 04:01:03,953 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/3255711fd2fb41868da8a0dffcbbb6c5 2023-02-12 04:01:03,954 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,954 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/81605c0591ab4ce591a77144516216a7 2023-02-12 04:01:03,955 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/19d9f0fe771f4e31832b51797912c11c.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,956 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,957 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/8d6d0d0405424b6bb655b3383e05c4f8 2023-02-12 04:01:03,957 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/3255711fd2fb41868da8a0dffcbbb6c5.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,958 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,958 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d001b707ba4e4e7189b2b81c6e20a812 2023-02-12 04:01:03,959 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/81605c0591ab4ce591a77144516216a7.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,960 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,960 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/d853e69474094f10a25af38df7dadeb5 2023-02-12 04:01:03,961 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/8d6d0d0405424b6bb655b3383e05c4f8.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,962 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,962 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/cf2/e35dd2eab4ee421abccb2922161727b2 2023-02-12 04:01:03,962 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/d001b707ba4e4e7189b2b81c6e20a812.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,963 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,965 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/d853e69474094f10a25af38df7dadeb5.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,965 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/3d0c2ff996cdb076c77801003013a683/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,966 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/recovered.edits/85.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f/recovered.edits/85.seqid 2023-02-12 04:01:03,967 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,967 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/2405ca844e0b3394a8e057a0063d11f6/cf2/e35dd2eab4ee421abccb2922161727b2.14a5b26b7821b29d8e7405e41ed6325f 2023-02-12 04:01:03,968 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/97027828354d43d9b3dbceaaaa5f21e2 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf1/97027828354d43d9b3dbceaaaa5f21e2 2023-02-12 04:01:03,970 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/0d41704b07944ca2acbe5c6a77de00b2 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf1/0d41704b07944ca2acbe5c6a77de00b2 2023-02-12 04:01:03,971 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/860ee2da5a80464b8b3d35e6367ac4f3 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/cf2/860ee2da5a80464b8b3d35e6367ac4f3 2023-02-12 04:01:03,973 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/25018ef341fa4cca9ca95941e679b665 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/cf2/25018ef341fa4cca9ca95941e679b665 2023-02-12 04:01:03,974 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/recovered.edits/90.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683/recovered.edits/90.seqid 2023-02-12 04:01:03,978 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:03,980 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/recovered.edits/90.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6/recovered.edits/90.seqid 2023-02-12 04:01:03,981 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitWithoutPONR/2405ca844e0b3394a8e057a0063d11f6 2023-02-12 04:01:03,981 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-12 04:01:03,984 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:03,987 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-12 04:01:03,989 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-12 04:01:03,991 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:03,991 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-12 04:01:03,991 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174463991"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:03,991 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174463991"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:03,991 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174463991"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:03,993 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-12 04:01:03,993 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 14a5b26b7821b29d8e7405e41ed6325f, NAME => 'testSplitWithoutPONR,,1676174457155.14a5b26b7821b29d8e7405e41ed6325f.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 2405ca844e0b3394a8e057a0063d11f6, NAME => 'testSplitWithoutPONR,,1676174459570.2405ca844e0b3394a8e057a0063d11f6.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 3d0c2ff996cdb076c77801003013a683, NAME => 'testSplitWithoutPONR,41,1676174459570.3d0c2ff996cdb076c77801003013a683.', STARTKEY => '41', ENDKEY => ''}] 2023-02-12 04:01:03,993 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-12 04:01:03,993 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676174463993"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:03,995 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-12 04:01:03,997 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:03,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 74 msec 2023-02-12 04:01:04,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-12 04:01:04,028 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-12 04:01:04,041 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=395 (was 389) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:60164 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:54008 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:60186 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c2e965c-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:53996 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:60170 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:53934 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:39804 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:38728 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:53988 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:60132 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:40044 [Waiting for operation #37] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:39284 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:40096 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:39280 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:38278 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1474640581_15 at /127.0.0.1:53986 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=861 (was 844) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=196 (was 204), ProcessCount=165 (was 165), AvailableMemoryMB=4154 (was 4279) 2023-02-12 04:01:04,056 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=395, OpenFileDescriptor=861, MaxFileDescriptor=60000, SystemLoadAverage=196, ProcessCount=165, AvailableMemoryMB=4153 2023-02-12 04:01:04,057 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:01:04,059 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-12 04:01:04,059 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-12 04:01:04,061 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:04,063 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 04:01:04,064 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 04:01:04,065 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:04,066 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b empty. 2023-02-12 04:01:04,066 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:04,066 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-12 04:01:04,077 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-12 04:01:04,079 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0dc25aecee603c971d1e12ebe66ec89b, NAME => 'testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp 2023-02-12 04:01:04,087 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:04,087 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 0dc25aecee603c971d1e12ebe66ec89b, disabling compactions & flushes 2023-02-12 04:01:04,087 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:04,087 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:04,087 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. after waiting 0 ms 2023-02-12 04:01:04,087 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:04,087 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:04,087 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 0dc25aecee603c971d1e12ebe66ec89b: Waiting for close lock at 1676174464087Disabling compacts and flushes for region at 1676174464087Disabling writes for close at 1676174464087Writing region close event to WAL at 1676174464087Closed at 1676174464087 2023-02-12 04:01:04,089 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 04:01:04,090 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174464090"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174464090"}]},"ts":"1676174464090"} 2023-02-12 04:01:04,092 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 04:01:04,093 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 04:01:04,093 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174464093"}]},"ts":"1676174464093"} 2023-02-12 04:01:04,095 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-12 04:01:04,098 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-12 04:01:04,099 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-12 04:01:04,100 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-12 04:01:04,100 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-12 04:01:04,100 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-12 04:01:04,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN}] 2023-02-12 04:01:04,102 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:04,103 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,43977,1676174449184; forceNewPlan=false, retain=false 2023-02-12 04:01:04,253 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-12 04:01:04,254 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=0dc25aecee603c971d1e12ebe66ec89b, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,254 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174464254"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174464254"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174464254"}]},"ts":"1676174464254"} 2023-02-12 04:01:04,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:01:04,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:04,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0dc25aecee603c971d1e12ebe66ec89b, NAME => 'testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:01:04,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:01:04,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:04,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:04,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:04,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:04,414 INFO [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:04,415 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1 2023-02-12 04:01:04,415 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1 2023-02-12 04:01:04,416 INFO [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0dc25aecee603c971d1e12ebe66ec89b columnFamilyName cf1 2023-02-12 04:01:04,416 INFO [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(310): Store=0dc25aecee603c971d1e12ebe66ec89b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:04,416 INFO [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:04,417 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2 2023-02-12 04:01:04,417 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2 2023-02-12 04:01:04,418 INFO [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0dc25aecee603c971d1e12ebe66ec89b columnFamilyName cf2 2023-02-12 04:01:04,418 INFO [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(310): Store=0dc25aecee603c971d1e12ebe66ec89b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:04,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:04,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:04,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:01:04,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:04,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 04:01:04,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 0dc25aecee603c971d1e12ebe66ec89b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64326430, jitterRate=-0.04146149754524231}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:01:04,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 0dc25aecee603c971d1e12ebe66ec89b: Running coprocessor pre-open hook at 1676174464412Writing region info on filesystem at 1676174464412Initializing all the Stores at 1676174464414 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174464414Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174464414Cleaning up temporary data from old regions at 1676174464420 (+6 ms)Cleaning up detritus from prior splits at 1676174464420Running coprocessor post-open hooks at 1676174464424 (+4 ms)Region opened successfully at 1676174464424 2023-02-12 04:01:04,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b., pid=43, masterSystemTime=1676174464408 2023-02-12 04:01:04,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:04,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:04,427 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=0dc25aecee603c971d1e12ebe66ec89b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,427 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174464427"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174464427"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174464427"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174464427"}]},"ts":"1676174464427"} 2023-02-12 04:01:04,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-12 04:01:04,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 in 173 msec 2023-02-12 04:01:04,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-12 04:01:04,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN in 331 msec 2023-02-12 04:01:04,434 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 04:01:04,434 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174464434"}]},"ts":"1676174464434"} 2023-02-12 04:01:04,435 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-12 04:01:04,437 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 04:01:04,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 377 msec 2023-02-12 04:01:04,571 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:04,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-12 04:01:04,573 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-12 04:01:04,573 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:04,573 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-12 04:01:04,573 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-12 04:01:04,574 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,574 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,575 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,575 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,575 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,575 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,575 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,575 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:04,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:04,576 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,576 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:04,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:04,577 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:04,577 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:04,577 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:04,577 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:04,577 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. started... 2023-02-12 04:01:04,578 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0dc25aecee603c971d1e12ebe66ec89b 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-12 04:01:04,578 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,579 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,579 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:04,579 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,579 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,579 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:04,590 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/b226d99488c3445cb25f1f8c4804e97d 2023-02-12 04:01:04,607 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/2c7c1221ab184268b7d7515c67a3a6a0 2023-02-12 04:01:04,613 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/b226d99488c3445cb25f1f8c4804e97d as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/b226d99488c3445cb25f1f8c4804e97d 2023-02-12 04:01:04,618 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/b226d99488c3445cb25f1f8c4804e97d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:01:04,619 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/2c7c1221ab184268b7d7515c67a3a6a0 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/2c7c1221ab184268b7d7515c67a3a6a0 2023-02-12 04:01:04,624 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/2c7c1221ab184268b7d7515c67a3a6a0, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:01:04,624 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 0dc25aecee603c971d1e12ebe66ec89b in 46ms, sequenceid=6, compaction requested=false 2023-02-12 04:01:04,624 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-12 04:01:04,625 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0dc25aecee603c971d1e12ebe66ec89b: 2023-02-12 04:01:04,625 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:04,625 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:04,625 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:04,625 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,625 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:04,625 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:04,626 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,626 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:04,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:04,627 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,627 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:04,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:04,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:04,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:04,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:04,628 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-12 04:01:04,628 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-12 04:01:04,628 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70e422ed[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:04,628 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,629 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,629 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,629 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,629 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:04,629 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:04,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,629 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,629 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:04,630 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:04,629 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:04,630 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:04,630 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:04,630 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:04,630 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:04,630 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:04,630 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:04,630 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:04,631 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:04,631 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,631 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:04,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,631 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:04,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:04,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:04,631 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:04,631 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:04,631 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:04,631 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:04,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:04,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:04,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:01:04,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-12 04:01:04,633 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-12 04:01:04,633 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:04,633 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:04,633 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-12 04:01:04,634 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:04,634 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,634 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,635 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,635 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:04,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:04,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:04,635 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,635 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:04,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,635 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,635 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:04,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:04,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:04,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:04,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:04,635 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:04,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:04,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:04,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,636 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:04,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:04,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:04,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,638 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,638 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:04,638 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:04,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,640 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,640 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,640 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,640 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:04,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,640 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,640 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,640 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-12 04:01:04,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:04,640 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:04,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:04,641 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:04,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,640 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:04,640 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,641 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:04,640 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:04,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:04,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:04,641 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:04,641 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,642 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,642 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,642 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:04,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:04,642 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,642 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,642 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,642 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,742 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:04,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:04,761 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:04,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-12 04:01:04,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-12 04:01:04,763 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-12 04:01:04,763 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:04,763 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-12 04:01:04,763 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-12 04:01:04,763 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,763 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,764 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,764 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,764 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,764 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,765 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,765 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:04,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,765 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,765 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,765 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:04,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:04,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,765 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:04,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:04,766 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:04,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:04,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,766 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:04,766 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:04,766 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:04,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:04,766 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:04,766 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:04,766 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:04,766 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:04,766 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:04,766 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:04,766 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:04,766 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:04,766 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:04,766 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:04,766 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:04,767 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:04,767 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:04,767 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:04,767 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:04,767 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:04,767 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:04,767 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:04,767 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. started... 2023-02-12 04:01:04,767 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0dc25aecee603c971d1e12ebe66ec89b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:01:04,767 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,768 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,768 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,768 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:04,768 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,768 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:04,777 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/587ab1e026244110a0337602d96b400b 2023-02-12 04:01:04,790 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/a87311e55dd84b8ca921d2a5fb51c38c 2023-02-12 04:01:04,796 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/587ab1e026244110a0337602d96b400b as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/587ab1e026244110a0337602d96b400b 2023-02-12 04:01:04,800 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/587ab1e026244110a0337602d96b400b, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:01:04,801 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/a87311e55dd84b8ca921d2a5fb51c38c as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/a87311e55dd84b8ca921d2a5fb51c38c 2023-02-12 04:01:04,806 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/a87311e55dd84b8ca921d2a5fb51c38c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:01:04,807 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0dc25aecee603c971d1e12ebe66ec89b in 40ms, sequenceid=19, compaction requested=false 2023-02-12 04:01:04,807 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0dc25aecee603c971d1e12ebe66ec89b: 2023-02-12 04:01:04,807 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:04,807 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:04,807 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:04,807 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,807 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:04,807 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:04,808 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,809 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:04,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:04,809 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,809 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:04,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:04,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:04,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:04,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:04,811 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-12 04:01:04,811 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60569769[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:04,811 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-12 04:01:04,811 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,812 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,812 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,812 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,812 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,812 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:04,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,812 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:04,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,812 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:04,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,813 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:04,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,813 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:04,813 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:04,813 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:04,813 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:04,813 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:04,813 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:04,815 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:04,815 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:04,815 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:04,815 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:04,815 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:04,815 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,815 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:04,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,815 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:04,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:04,815 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:04,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:04,815 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:04,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:04,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:04,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:04,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:04,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:04,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:01:04,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-12 04:01:04,819 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-12 04:01:04,819 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:04,819 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:04,819 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-12 04:01:04,819 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:04,820 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,820 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,820 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,820 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:04,820 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:04,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:04,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,820 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:04,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:04,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,821 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:04,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:04,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:04,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:04,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:04,821 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:04,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:04,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:04,822 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:04,822 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:04,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:04,824 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:04,824 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,824 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,826 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,826 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,826 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,826 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,826 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:04,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,826 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,826 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:04,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,827 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,826 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-12 04:01:04,827 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:04,826 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:04,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:04,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,826 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:04,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:04,827 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,827 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,827 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:04,827 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:04,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:04,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:04,827 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:04,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:04,828 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:04,828 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:04,828 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,828 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,828 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,928 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:04,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:04,944 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:04,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-12 04:01:04,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-12 04:01:04,946 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-12 04:01:04,946 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:04,946 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-12 04:01:04,946 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-12 04:01:04,946 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,946 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,947 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,947 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,947 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,947 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:04,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,947 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,948 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:04,949 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:04,949 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:04,949 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:04,949 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:04,949 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:04,949 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:04,949 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:04,949 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:04,949 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:04,949 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:04,949 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:04,949 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:04,949 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:04,949 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:04,949 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:04,949 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:04,949 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:04,949 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:04,949 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:04,949 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:04,949 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:04,950 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:04,950 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:04,950 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:04,950 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. started... 2023-02-12 04:01:04,950 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0dc25aecee603c971d1e12ebe66ec89b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:01:04,950 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,950 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,952 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,952 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:04,952 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,952 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:04,962 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/c944f93481a14a938f74fa827d2794ca 2023-02-12 04:01:04,981 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/30bc1796b30540cf992d5c27f95594ed 2023-02-12 04:01:04,986 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/c944f93481a14a938f74fa827d2794ca as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/c944f93481a14a938f74fa827d2794ca 2023-02-12 04:01:04,991 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/c944f93481a14a938f74fa827d2794ca, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:01:04,992 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/30bc1796b30540cf992d5c27f95594ed as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/30bc1796b30540cf992d5c27f95594ed 2023-02-12 04:01:04,996 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/30bc1796b30540cf992d5c27f95594ed, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:01:04,997 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0dc25aecee603c971d1e12ebe66ec89b in 47ms, sequenceid=32, compaction requested=true 2023-02-12 04:01:04,997 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0dc25aecee603c971d1e12ebe66ec89b: 2023-02-12 04:01:04,997 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:04,997 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:04,998 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:04,998 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,998 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:04,998 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:04,999 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,999 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:04,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:04,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:04,999 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:04,999 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:04,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:05,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:05,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:05,001 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-12 04:01:05,001 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1eb54255[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:05,001 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-12 04:01:05,001 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,002 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,002 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,002 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,002 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,002 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:05,002 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:05,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,002 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:05,002 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:05,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,002 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:05,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,002 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:05,002 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:05,002 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:05,003 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:05,003 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:05,003 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:05,003 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,003 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:05,004 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:05,004 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,004 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:05,004 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:05,004 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,004 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:05,004 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:05,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:05,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:05,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:05,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:05,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:01:05,007 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-12 04:01:05,007 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-12 04:01:05,008 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:05,008 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:05,008 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-12 04:01:05,008 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:05,009 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,009 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,009 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,009 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,009 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,009 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,009 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:05,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:05,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,010 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:05,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,010 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,011 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:05,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,012 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:05,012 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,012 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,023 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,023 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,023 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,023 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,024 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-12 04:01:05,024 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:05,024 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,024 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,024 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,023 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,024 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:05,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,024 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,024 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:05,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,024 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,025 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,025 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,025 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,025 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,025 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,025 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,025 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,025 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,125 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:05,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:05,143 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:05,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-12 04:01:05,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-12 04:01:05,144 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-12 04:01:05,144 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:05,144 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-12 04:01:05,145 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-12 04:01:05,145 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,145 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,146 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,146 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,146 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,146 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,146 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,146 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:05,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:05,147 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:05,147 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:05,147 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:05,147 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:05,148 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:05,148 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:05,148 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. started... 2023-02-12 04:01:05,148 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0dc25aecee603c971d1e12ebe66ec89b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:01:05,149 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,149 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,149 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,149 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:05,149 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,149 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:05,563 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/2644ad6f0d634d25a1cd45968f4a3f23 2023-02-12 04:01:05,582 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/eed9fa1e416247c5be591ec1a6ddf0ec 2023-02-12 04:01:05,587 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/2644ad6f0d634d25a1cd45968f4a3f23 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2644ad6f0d634d25a1cd45968f4a3f23 2023-02-12 04:01:05,592 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2644ad6f0d634d25a1cd45968f4a3f23, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:01:05,593 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/eed9fa1e416247c5be591ec1a6ddf0ec as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/eed9fa1e416247c5be591ec1a6ddf0ec 2023-02-12 04:01:05,598 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/eed9fa1e416247c5be591ec1a6ddf0ec, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:01:05,599 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0dc25aecee603c971d1e12ebe66ec89b in 451ms, sequenceid=45, compaction requested=true 2023-02-12 04:01:05,599 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0dc25aecee603c971d1e12ebe66ec89b: 2023-02-12 04:01:05,599 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:05,599 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:05,599 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:05,599 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,599 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:05,599 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:05,600 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,600 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:05,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:05,601 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,601 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:05,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:05,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:05,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:05,602 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-12 04:01:05,602 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2deb2dc1[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:05,602 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-12 04:01:05,602 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,603 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,603 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,603 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,603 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:05,603 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:05,603 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:05,603 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,603 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:05,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,603 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:05,604 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:05,604 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:05,604 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:05,604 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:05,604 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:05,604 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:05,604 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,604 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:05,604 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:05,605 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,605 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:05,605 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:05,605 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,605 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:05,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:05,605 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:05,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:05,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:05,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:05,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:01:05,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-12 04:01:05,608 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-12 04:01:05,608 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:05,608 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:05,608 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-12 04:01:05,608 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:05,609 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,609 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,609 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,609 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,609 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,609 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,609 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:05,609 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:05,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,610 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:05,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,610 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,611 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,611 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:05,612 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,613 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,613 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,613 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,614 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,613 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,613 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,614 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,614 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,614 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,614 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,614 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-12 04:01:05,614 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:05,614 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,614 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,614 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,614 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,614 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:05,614 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,615 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:05,615 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,615 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,615 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,715 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:05,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:05,731 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:05,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-12 04:01:05,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-12 04:01:05,733 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-12 04:01:05,733 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:05,733 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-12 04:01:05,733 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-12 04:01:05,734 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,734 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,735 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,735 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,735 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,735 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,735 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,735 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:05,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:05,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:05,736 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:05,737 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:05,737 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:05,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:05,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:05,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:05,737 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:05,737 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. started... 2023-02-12 04:01:05,737 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:05,737 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0dc25aecee603c971d1e12ebe66ec89b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:01:05,738 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,738 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,739 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,739 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,739 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:05,739 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:05,750 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/d07f9eb8feeb4de381f15cf69fadacfd 2023-02-12 04:01:05,766 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/03c07a9889c54f9699de00002a5cb893 2023-02-12 04:01:05,770 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/d07f9eb8feeb4de381f15cf69fadacfd as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/d07f9eb8feeb4de381f15cf69fadacfd 2023-02-12 04:01:05,774 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/d07f9eb8feeb4de381f15cf69fadacfd, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:01:05,775 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/03c07a9889c54f9699de00002a5cb893 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/03c07a9889c54f9699de00002a5cb893 2023-02-12 04:01:05,779 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/03c07a9889c54f9699de00002a5cb893, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:01:05,780 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0dc25aecee603c971d1e12ebe66ec89b in 43ms, sequenceid=58, compaction requested=true 2023-02-12 04:01:05,780 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0dc25aecee603c971d1e12ebe66ec89b: 2023-02-12 04:01:05,780 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:05,780 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:05,780 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:05,780 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,780 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:05,780 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:05,781 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,781 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:05,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:05,782 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,782 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:05,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:05,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:05,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:05,783 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-12 04:01:05,783 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c4991db[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:05,783 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-12 04:01:05,783 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,784 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,784 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,784 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,784 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:05,784 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,784 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:05,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,784 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:05,784 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:05,784 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:05,784 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:05,784 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:05,784 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:05,784 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:05,784 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:05,786 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:05,786 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,786 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:05,786 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:05,786 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:05,786 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,786 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,786 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:05,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:05,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:05,786 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,786 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:05,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:05,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:05,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:05,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:01:05,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-12 04:01:05,789 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-12 04:01:05,789 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:05,789 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:05,789 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-12 04:01:05,789 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:05,790 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,790 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:05,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:05,790 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,790 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,790 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,790 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,791 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,791 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:05,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,791 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:05,792 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,793 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,793 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,793 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,795 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,795 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,795 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,795 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,795 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,795 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,795 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,795 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-12 04:01:05,795 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:05,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,795 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,795 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,795 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,795 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,796 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:05,796 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,796 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:05,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,796 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,796 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,796 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,796 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,796 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,796 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,896 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:05,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:05,914 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:05,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-12 04:01:05,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-12 04:01:05,916 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-12 04:01:05,916 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:05,916 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-12 04:01:05,916 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-12 04:01:05,917 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,917 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,917 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,917 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,917 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,917 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,918 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,918 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:05,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,918 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,918 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,918 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:05,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,918 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:05,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:05,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,919 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:05,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:05,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,919 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-12 04:01:05,919 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:05,919 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:05,919 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:05,919 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:05,919 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:05,919 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:05,919 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:05,919 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. started... 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:05,919 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:05,919 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0dc25aecee603c971d1e12ebe66ec89b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:01:05,920 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,921 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,921 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,921 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:05,921 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,921 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:05,928 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/2dfc9429c8e449e481b3bbe13ce0f1ae 2023-02-12 04:01:05,940 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/1722a28298d54844a883882c3886f26c 2023-02-12 04:01:05,945 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/2dfc9429c8e449e481b3bbe13ce0f1ae as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2dfc9429c8e449e481b3bbe13ce0f1ae 2023-02-12 04:01:05,950 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2dfc9429c8e449e481b3bbe13ce0f1ae, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:01:05,951 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/1722a28298d54844a883882c3886f26c as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/1722a28298d54844a883882c3886f26c 2023-02-12 04:01:05,956 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/1722a28298d54844a883882c3886f26c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:01:05,957 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0dc25aecee603c971d1e12ebe66ec89b in 38ms, sequenceid=71, compaction requested=true 2023-02-12 04:01:05,957 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0dc25aecee603c971d1e12ebe66ec89b: 2023-02-12 04:01:05,957 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:05,957 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:05,957 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:05,957 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,958 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-12 04:01:05,958 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-12 04:01:05,959 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,959 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:05,959 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:05,959 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:05,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-12 04:01:05,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2870ace4[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:05,961 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-12 04:01:05,961 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,961 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,962 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:05,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,962 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:05,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,962 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:05,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,962 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:05,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,962 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:05,962 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:01:05,962 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:05,962 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:05,962 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-12 04:01:05,962 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:05,963 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,963 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:05,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:05,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:05,963 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:05,963 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,963 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,963 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-12 04:01:05,963 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,963 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:05,963 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:05,963 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-12 04:01:05,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:05,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:01:05,966 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-12 04:01:05,966 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-12 04:01:05,966 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:05,966 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:05,966 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-12 04:01:05,966 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:05,968 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,968 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,968 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,968 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,968 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,968 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:05,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,968 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:05,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,969 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:05,970 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:05,970 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:05,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-12 04:01:05,972 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,973 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,973 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,976 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,976 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,976 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,976 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,976 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,976 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,976 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:05,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,976 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-12 04:01:05,976 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:05,976 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:05,976 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:05,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:05,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,977 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:05,976 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:05,977 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:05,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,976 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:05,977 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-12 04:01:05,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:05,977 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:05,977 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,977 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:05,977 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:05,977 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-12 04:01:05,977 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-12 04:01:06,077 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:06,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:06,340 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-12 04:01:06,341 DEBUG [Listener at localhost.localdomain/35207] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:06,342 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,343 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,343 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,343 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,344 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:06,345 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:06,345 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:06,592 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,593 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:06,593 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:06,593 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:06,593 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-12 04:01:06,593 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:06,593 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:06,593 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:06,593 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:06,594 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:06,595 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-12 04:01:06,595 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:06,596 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:06,596 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:06,596 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:06,596 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:06,596 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:06,596 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:06,597 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:06,597 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:06,597 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:06,597 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:06,597 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:06,598 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:06,598 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,598 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:06,598 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,598 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,598 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,598 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-12 04:01:06,599 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:06,599 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:06,600 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174496600 2023-02-12 04:01:06,601 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:06,601 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,603 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:06,604 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:06,604 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:06,604 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174526604 2023-02-12 04:01:06,604 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174586604 2023-02-12 04:01:06,604 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:06,604 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:06,607 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:06,608 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:06,608 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:06,854 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,854 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:06,854 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:06,854 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:06,855 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:06,855 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:06,855 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:06,855 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:06,855 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:06,856 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:06,857 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-12 04:01:06,857 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:06,857 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:06,857 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:06,857 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:06,858 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:06,858 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:06,858 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:06,858 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:06,858 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:06,859 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:06,859 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:06,859 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:06,859 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:06,859 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,859 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:06,860 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,860 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,860 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,860 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-12 04:01:06,860 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:06,861 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:06,861 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174496861 2023-02-12 04:01:06,862 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,862 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:06,865 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:06,865 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:06,865 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:06,865 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174526865 2023-02-12 04:01:06,865 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174586865 2023-02-12 04:01:06,866 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:06,866 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:06,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN}] 2023-02-12 04:01:06,868 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:06,868 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:06,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:06,868 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:07,116 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,116 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:07,116 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:07,116 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:07,116 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:07,116 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:07,116 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:07,117 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:07,117 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:07,117 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:07,118 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-12 04:01:07,119 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:07,119 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:07,119 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:07,120 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:07,120 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:07,120 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:07,121 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:07,121 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:07,121 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:07,122 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:07,122 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:07,123 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:07,123 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:07,124 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,124 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:07,124 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,124 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,124 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,124 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-12 04:01:07,125 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:07,125 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:07,125 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174497125 2023-02-12 04:01:07,127 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:07,127 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,129 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:07,129 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:07,129 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:07,129 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174527129 2023-02-12 04:01:07,129 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174587129 2023-02-12 04:01:07,129 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-12 04:01:07,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN}] 2023-02-12 04:01:07,130 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:07,132 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:07,133 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:07,133 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174467133"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174467133"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174467133"}]},"ts":"1676174467133"} 2023-02-12 04:01:07,134 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:01:07,134 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:07,134 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:07,134 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:07,135 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:07,379 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,380 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:07,380 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:07,380 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:07,381 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:07,381 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:07,381 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:07,381 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:07,381 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:07,382 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:07,383 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-12 04:01:07,383 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-12 04:01:07,383 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:07,383 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:07,384 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:07,384 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:07,384 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:07,384 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:07,384 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:07,384 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:07,385 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:07,385 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:07,385 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:07,385 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:07,385 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:07,386 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,386 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:07,386 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:07,386 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,386 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,386 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,386 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:07,386 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:07,387 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-12 04:01:07,387 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:07,388 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:07,388 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b to restore RIT 2023-02-12 04:01:07,388 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174497388 2023-02-12 04:01:07,389 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:07,389 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:07,391 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:07,392 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSING, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:07,392 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:07,392 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174527392 2023-02-12 04:01:07,392 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174587392 2023-02-12 04:01:07,392 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-12 04:01:07,392 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:07,392 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174467392"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174467392"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174467392"}]},"ts":"1676174467392"} 2023-02-12 04:01:07,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:01:07,394 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:07,547 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:07,547 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 0dc25aecee603c971d1e12ebe66ec89b, disabling compactions & flushes 2023-02-12 04:01:07,547 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:07,547 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:07,547 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. after waiting 0 ms 2023-02-12 04:01:07,547 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:07,548 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 0dc25aecee603c971d1e12ebe66ec89b 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-12 04:01:07,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/2425e2074a554800a1cb2b054f37c4ee 2023-02-12 04:01:07,569 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/984af9a12e924e769e60064568763d89 2023-02-12 04:01:07,574 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/2425e2074a554800a1cb2b054f37c4ee as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2425e2074a554800a1cb2b054f37c4ee 2023-02-12 04:01:07,578 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2425e2074a554800a1cb2b054f37c4ee, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:01:07,579 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/984af9a12e924e769e60064568763d89 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/984af9a12e924e769e60064568763d89 2023-02-12 04:01:07,583 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/984af9a12e924e769e60064568763d89, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:01:07,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 0dc25aecee603c971d1e12ebe66ec89b in 37ms, sequenceid=82, compaction requested=true 2023-02-12 04:01:07,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-12 04:01:07,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:01:07,588 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:07,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 0dc25aecee603c971d1e12ebe66ec89b: Waiting for close lock at 1676174467547Running coprocessor pre-close hooks at 1676174467547Disabling compacts and flushes for region at 1676174467547Disabling writes for close at 1676174467547Obtaining lock to block concurrent updates at 1676174467548 (+1 ms)Preparing flush snapshotting stores in 0dc25aecee603c971d1e12ebe66ec89b at 1676174467548Finished memstore snapshotting testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676174467548Flushing stores of testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. at 1676174467548Flushing 0dc25aecee603c971d1e12ebe66ec89b/cf1: creating writer at 1676174467549 (+1 ms)Flushing 0dc25aecee603c971d1e12ebe66ec89b/cf1: appending metadata at 1676174467551 (+2 ms)Flushing 0dc25aecee603c971d1e12ebe66ec89b/cf1: closing flushed file at 1676174467551Flushing 0dc25aecee603c971d1e12ebe66ec89b/cf2: creating writer at 1676174467561 (+10 ms)Flushing 0dc25aecee603c971d1e12ebe66ec89b/cf2: appending metadata at 1676174467563 (+2 ms)Flushing 0dc25aecee603c971d1e12ebe66ec89b/cf2: closing flushed file at 1676174467563Flushing 0dc25aecee603c971d1e12ebe66ec89b/cf1: reopening flushed file at 1676174467574 (+11 ms)Flushing 0dc25aecee603c971d1e12ebe66ec89b/cf2: reopening flushed file at 1676174467579 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 0dc25aecee603c971d1e12ebe66ec89b in 37ms, sequenceid=82, compaction requested=true at 1676174467584 (+5 ms)Writing region close event to WAL at 1676174467585 (+1 ms)Running coprocessor post-close hooks at 1676174467588 (+3 ms)Closed at 1676174467588 2023-02-12 04:01:07,590 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:07,591 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED 2023-02-12 04:01:07,591 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174467591"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174467591"}]},"ts":"1676174467591"} 2023-02-12 04:01:07,593 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:07,593 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:07,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:07,593 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:07,642 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,642 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:07,642 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:07,642 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:07,643 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:07,643 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:07,644 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:07,644 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:07,644 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:07,644 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:07,645 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-12 04:01:07,646 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-12 04:01:07,646 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-12 04:01:07,646 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:07,646 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:07,647 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:07,647 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:07,647 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:07,647 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:07,648 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:07,648 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:07,648 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:07,648 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:07,649 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:07,649 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:07,649 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:07,650 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,650 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:07,650 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:07,650 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:07,650 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,650 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,650 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,650 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:07,650 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:07,651 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:07,651 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-12 04:01:07,652 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:07,652 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:07,652 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b to restore RIT 2023-02-12 04:01:07,653 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174497653 2023-02-12 04:01:07,653 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:07,653 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:07,655 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:07,656 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:07,656 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:07,656 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174527656 2023-02-12 04:01:07,656 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174587656 2023-02-12 04:01:07,656 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-12 04:01:07,656 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:07,656 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174467656"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174467656"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174467656"}]},"ts":"1676174467656"} 2023-02-12 04:01:07,658 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:07,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-12 04:01:07,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 in 264 msec 2023-02-12 04:01:07,661 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:07,661 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:07,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:07,662 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:07,906 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,906 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:07,906 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:07,907 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:07,909 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:07,909 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:07,909 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:07,909 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:07,909 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:07,910 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:07,911 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-12 04:01:07,911 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-12 04:01:07,911 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:07,911 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:07,912 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:07,912 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:07,912 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:07,913 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:07,913 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:07,913 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:07,913 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:07,914 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:07,914 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:07,914 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:07,914 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:07,915 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:07,916 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,916 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:07,916 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:07,916 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:07,916 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,916 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,916 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,916 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:07,917 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:07,917 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-12 04:01:07,917 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:07,918 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:07,918 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b to restore RIT 2023-02-12 04:01:07,920 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174497920 2023-02-12 04:01:07,920 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:07,921 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:07,922 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:07,923 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:07,923 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:07,923 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174527923 2023-02-12 04:01:07,923 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174587923 2023-02-12 04:01:07,923 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-12 04:01:07,923 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:07,925 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:07,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-12 04:01:07,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN in 796 msec 2023-02-12 04:01:07,930 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:07,931 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:07,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:07,931 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:08,173 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:08,174 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:08,174 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:08,174 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:08,175 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:08,175 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:08,175 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:08,175 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:08,175 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:08,176 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:08,176 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-12 04:01:08,177 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:01:08,177 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:08,177 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:08,177 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:08,177 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:08,177 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:08,178 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:08,178 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:08,178 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:08,178 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:08,179 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:08,179 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:08,179 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:08,179 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:08,180 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:08,180 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:08,180 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:08,180 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:08,180 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:08,180 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:08,180 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:08,180 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-12 04:01:08,181 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:08,182 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:08,191 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174498191 2023-02-12 04:01:08,192 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:08,192 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:08,195 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:08,195 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:08,195 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:08,195 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174528195 2023-02-12 04:01:08,196 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174588196 2023-02-12 04:01:08,196 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:08,196 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:08,196 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:08,196 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:08,201 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:08,210 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=0dc25aecee603c971d1e12ebe66ec89b, threads=14 2023-02-12 04:01:08,211 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/03c07a9889c54f9699de00002a5cb893 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,211 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/1722a28298d54844a883882c3886f26c for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,211 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/2c7c1221ab184268b7d7515c67a3a6a0 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,211 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/30bc1796b30540cf992d5c27f95594ed for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,211 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/984af9a12e924e769e60064568763d89 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,219 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/a87311e55dd84b8ca921d2a5fb51c38c for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,219 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/eed9fa1e416247c5be591ec1a6ddf0ec for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,219 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2644ad6f0d634d25a1cd45968f4a3f23 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,220 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2425e2074a554800a1cb2b054f37c4ee for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,220 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/587ab1e026244110a0337602d96b400b for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,220 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/b226d99488c3445cb25f1f8c4804e97d for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,220 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2dfc9429c8e449e481b3bbe13ce0f1ae for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,223 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/c944f93481a14a938f74fa827d2794ca for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,227 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/d07f9eb8feeb4de381f15cf69fadacfd for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,403 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/587ab1e026244110a0337602d96b400b for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,435 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/03c07a9889c54f9699de00002a5cb893 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,435 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/1722a28298d54844a883882c3886f26c for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,439 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/b226d99488c3445cb25f1f8c4804e97d for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,460 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/30bc1796b30540cf992d5c27f95594ed for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,466 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/c944f93481a14a938f74fa827d2794ca for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,479 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2425e2074a554800a1cb2b054f37c4ee for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,479 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/984af9a12e924e769e60064568763d89 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,481 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/eed9fa1e416247c5be591ec1a6ddf0ec for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,481 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/d07f9eb8feeb4de381f15cf69fadacfd for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,482 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2644ad6f0d634d25a1cd45968f4a3f23 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,483 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/2c7c1221ab184268b7d7515c67a3a6a0 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,487 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2dfc9429c8e449e481b3bbe13ce0f1ae for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,489 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/a87311e55dd84b8ca921d2a5fb51c38c for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,489 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 0dc25aecee603c971d1e12ebe66ec89b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-12 04:01:08,526 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:08,527 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:08,527 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:08,527 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:08,537 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 04:01:08,696 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:08,696 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:08,696 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:08,696 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:08,697 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:08,697 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:08,697 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:08,697 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:08,697 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:08,698 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:08,699 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-12 04:01:08,699 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:01:08,699 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:08,699 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:08,700 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:08,700 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:08,700 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:08,700 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:08,700 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:08,700 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:08,701 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:08,701 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:08,701 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:08,701 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:08,702 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:08,702 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:08,703 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:08,703 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:08,703 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:08,703 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:08,703 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:08,703 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:08,703 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:08,703 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:08,703 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:08,704 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-12 04:01:08,704 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:08,705 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:08,705 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174498705 2023-02-12 04:01:08,706 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:08,706 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:08,708 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:08,709 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:08,709 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:08,709 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174528709 2023-02-12 04:01:08,709 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174588709 2023-02-12 04:01:08,709 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-12 04:01:08,718 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=0dc25aecee603c971d1e12ebe66ec89b, threads=14 2023-02-12 04:01:08,719 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/03c07a9889c54f9699de00002a5cb893 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,719 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/1722a28298d54844a883882c3886f26c for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,719 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/2c7c1221ab184268b7d7515c67a3a6a0 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,719 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/30bc1796b30540cf992d5c27f95594ed for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,719 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/984af9a12e924e769e60064568763d89 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,719 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/a87311e55dd84b8ca921d2a5fb51c38c for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,720 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/eed9fa1e416247c5be591ec1a6ddf0ec for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,720 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2425e2074a554800a1cb2b054f37c4ee for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,720 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2644ad6f0d634d25a1cd45968f4a3f23 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,720 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2dfc9429c8e449e481b3bbe13ce0f1ae for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,720 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/587ab1e026244110a0337602d96b400b for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,720 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/b226d99488c3445cb25f1f8c4804e97d for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,721 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/c944f93481a14a938f74fa827d2794ca for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,721 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/d07f9eb8feeb4de381f15cf69fadacfd for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,868 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/eed9fa1e416247c5be591ec1a6ddf0ec for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,879 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/1722a28298d54844a883882c3886f26c for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,896 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/984af9a12e924e769e60064568763d89 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,896 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/a87311e55dd84b8ca921d2a5fb51c38c for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,912 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/03c07a9889c54f9699de00002a5cb893 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,923 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2644ad6f0d634d25a1cd45968f4a3f23 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,933 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/587ab1e026244110a0337602d96b400b for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,935 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/c944f93481a14a938f74fa827d2794ca for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,935 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/30bc1796b30540cf992d5c27f95594ed for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,937 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/2c7c1221ab184268b7d7515c67a3a6a0 for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,943 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/b226d99488c3445cb25f1f8c4804e97d for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,945 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2dfc9429c8e449e481b3bbe13ce0f1ae for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:08,946 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2425e2074a554800a1cb2b054f37c4ee for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:09,244 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/d07f9eb8feeb4de381f15cf69fadacfd for region: 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:09,244 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 0dc25aecee603c971d1e12ebe66ec89b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-12 04:01:09,277 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:09,283 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/42a1d5b8ef83df4033d4e04086b0d8cc/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-12 04:01:09,284 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/dc6259f2524b5fadfb0532a05454680b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-12 04:01:09,284 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:09,284 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:09,284 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:09,459 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,460 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:09,460 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:09,460 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:09,460 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:09,460 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:09,460 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:09,460 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:09,460 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:09,461 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:09,462 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-12 04:01:09,462 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:01:09,462 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:09,462 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:09,463 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:09,463 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:09,463 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:09,463 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:09,463 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:09,464 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:09,464 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:09,464 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:09,465 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:09,465 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:09,465 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:09,465 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:09,466 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,466 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:09,466 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:09,468 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:09,468 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,468 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,468 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,468 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:09,469 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:09,469 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-12 04:01:09,469 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:09,470 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:09,474 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174499474 2023-02-12 04:01:09,475 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,475 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:09,478 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:09,478 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:09,478 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:09,478 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174529478 2023-02-12 04:01:09,478 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174589478 2023-02-12 04:01:09,478 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:09,481 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:09,485 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:09,485 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:09,485 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:09,729 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,729 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:09,729 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:09,729 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:09,730 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:09,730 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:09,730 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:09,730 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:09,730 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:09,730 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:09,731 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-12 04:01:09,731 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:01:09,731 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:09,731 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:09,732 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:09,732 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:09,732 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:09,732 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:09,733 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:09,733 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:09,733 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:09,733 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:09,733 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:09,734 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:09,734 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:09,734 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:09,735 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,735 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:09,735 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:09,735 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:09,735 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,735 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,735 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,736 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:09,736 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:09,736 DEBUG [Listener at localhost.localdomain/35207] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,736 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-12 04:01:09,737 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:09,737 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:09,737 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174499737 2023-02-12 04:01:09,738 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:09,740 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,742 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:09,742 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:09,742 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:09,742 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174529742 2023-02-12 04:01:09,743 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174589743 2023-02-12 04:01:09,743 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-12 04:01:09,743 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:09,753 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:09,753 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-12 04:01:09,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:09,754 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:09,994 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:09,994 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:09,994 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:09,994 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:09,994 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:09,994 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:09,994 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:09,995 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:09,995 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:09,995 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:09,997 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-12 04:01:09,997 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:01:09,997 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:09,997 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:09,997 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:09,997 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:09,998 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:09,998 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:09,998 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:09,998 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:09,998 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:09,998 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:09,999 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:09,999 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:09,999 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:09,999 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:10,000 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,000 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:10,001 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:10,001 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:10,001 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,001 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,001 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,001 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:10,001 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:10,001 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-12 04:01:10,002 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:10,002 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:10,002 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174500002 2023-02-12 04:01:10,005 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:10,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,007 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:10,008 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:10,008 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:10,008 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174530008 2023-02-12 04:01:10,008 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174590008 2023-02-12 04:01:10,008 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:10,008 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:10,011 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:10,011 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-12 04:01:10,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:10,011 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:10,259 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,259 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:10,259 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:10,259 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:10,259 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:10,260 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:10,260 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:10,260 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:10,260 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:10,261 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:10,261 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-12 04:01:10,262 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:01:10,262 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:10,262 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:10,262 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:10,262 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:10,263 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:10,263 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:10,263 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:10,263 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:10,263 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:10,264 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:10,264 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:10,264 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:10,264 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:10,265 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:10,265 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,266 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:10,266 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:10,266 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:10,266 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,266 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,266 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,266 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:10,266 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:10,266 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-12 04:01:10,267 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:10,268 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:10,268 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174500268 2023-02-12 04:01:10,269 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:10,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,271 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:10,271 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:10,271 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:10,272 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174530271 2023-02-12 04:01:10,272 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174590272 2023-02-12 04:01:10,272 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-12 04:01:10,272 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:10,274 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:10,275 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-12 04:01:10,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:10,275 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:10,522 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,522 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:10,522 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:10,522 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:10,522 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:10,522 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:10,522 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:10,522 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:10,523 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:10,523 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:10,524 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-12 04:01:10,524 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:01:10,524 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:10,524 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:10,525 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:10,525 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:10,525 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:10,525 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:10,525 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:10,526 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:10,526 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:10,526 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:10,526 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:10,526 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:10,526 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:10,526 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:10,527 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,527 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:10,527 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:10,527 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:10,527 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,527 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,527 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,527 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:10,527 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:10,528 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-12 04:01:10,528 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:10,529 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:10,529 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174500529 2023-02-12 04:01:10,530 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,530 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:10,533 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:10,533 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:10,533 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:10,533 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174530533 2023-02-12 04:01:10,533 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174590533 2023-02-12 04:01:10,533 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-12 04:01:10,533 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:10,537 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN] 2023-02-12 04:01:10,537 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:10,537 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-12 04:01:10,537 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:10,537 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:10,783 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,784 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:10,784 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:10,784 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:10,785 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:10,785 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:10,785 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:10,785 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:10,785 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:10,786 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:10,787 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-12 04:01:10,787 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:01:10,787 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:01:10,787 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-12 04:01:10,787 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:10,787 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:10,787 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:10,788 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:10,788 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:10,788 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:10,788 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:10,788 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:10,788 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:10,788 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:10,789 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:10,789 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:10,789 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:10,790 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,790 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:10,790 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:10,790 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:10,790 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:10,790 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,790 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,790 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,790 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:10,790 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:10,790 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:10,790 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-12 04:01:10,791 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:10,792 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:10,792 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b to restore RIT 2023-02-12 04:01:10,792 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174500792 2023-02-12 04:01:10,793 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:10,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:10,795 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:10,797 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:10,797 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:10,797 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:10,797 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174530797 2023-02-12 04:01:10,797 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174590797 2023-02-12 04:01:10,798 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-12 04:01:10,798 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:10,803 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:10,803 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-12 04:01:10,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:10,803 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:11,048 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,048 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:11,048 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:11,048 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:11,049 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:11,049 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:11,049 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:11,049 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:11,049 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:11,050 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:11,051 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-12 04:01:11,051 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-12 04:01:11,051 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:11,051 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:11,051 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:11,051 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:11,051 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:11,052 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:11,052 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:11,052 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:11,052 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:11,052 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:11,052 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:11,052 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:11,053 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:11,053 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,053 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:11,053 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:11,053 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,053 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,053 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,053 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:11,054 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-12 04:01:11,054 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:11,055 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:11,055 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b to restore RIT 2023-02-12 04:01:11,055 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174501055 2023-02-12 04:01:11,057 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:11,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,058 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:11,060 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:11,060 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:11,060 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:11,061 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174531061 2023-02-12 04:01:11,061 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174591061 2023-02-12 04:01:11,061 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:11,061 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:11,064 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:11,064 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-12 04:01:11,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:11,064 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:11,311 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,311 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:11,311 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:11,311 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:11,311 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:11,311 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:11,312 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:11,312 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:11,312 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:11,312 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:11,313 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-12 04:01:11,314 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-12 04:01:11,314 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:11,314 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:11,314 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:11,315 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:11,315 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:11,315 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:11,315 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:11,316 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:11,316 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:11,316 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:11,316 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:11,317 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:11,317 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:11,317 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,317 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:11,317 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:11,318 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,318 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,318 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,318 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:11,318 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-12 04:01:11,319 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:11,320 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:11,320 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b to restore RIT 2023-02-12 04:01:11,320 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174501320 2023-02-12 04:01:11,321 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:11,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,322 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:11,324 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:11,324 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:11,325 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:11,325 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174531325 2023-02-12 04:01:11,325 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174591325 2023-02-12 04:01:11,325 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:11,325 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:11,327 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:11,327 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-12 04:01:11,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:11,327 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:11,575 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,575 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:11,575 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:11,575 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:11,576 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:11,576 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:11,576 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:11,576 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:11,576 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:11,577 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:11,578 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-12 04:01:11,578 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-12 04:01:11,578 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:11,579 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:11,579 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:11,579 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:11,579 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:11,579 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:11,579 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:11,580 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:11,580 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:11,580 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:11,580 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:11,580 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:11,581 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:11,581 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,581 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:11,581 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:11,582 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,582 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,582 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,582 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:01:11,582 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-12 04:01:11,583 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:11,583 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:11,583 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b to restore RIT 2023-02-12 04:01:11,584 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174501584 2023-02-12 04:01:11,585 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,585 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:11,587 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:11,590 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:11,590 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:11,590 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:11,590 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174531590 2023-02-12 04:01:11,591 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174591591 2023-02-12 04:01:11,591 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-12 04:01:11,591 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:01:11,594 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b exec-time=5.2500 sec 2023-02-12 04:01:11,594 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:11,595 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,43977,1676174449184; forceNewPlan=false, retain=true 2023-02-12 04:01:11,745 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-12 04:01:11,747 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=0dc25aecee603c971d1e12ebe66ec89b, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:11,747 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174471746"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174471746"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174471746"}]},"ts":"1676174471746"} 2023-02-12 04:01:11,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:01:11,749 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:01:11,749 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:11,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:01:11,749 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:01:11,841 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:01:11,841 WARN [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-12 04:01:11,841 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:01:11,841 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:01:11,841 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:01:11,841 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:01:11,841 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:01:11,841 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:01:11,842 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:01:11,842 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:01:11,842 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:01:11,843 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-12 04:01:11,843 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:01:11,844 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:01:11,844 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:01:11,844 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:01:11,844 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:11,844 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:01:11,844 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:11,844 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:01:11,845 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:11,845 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:01:11,845 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:11,845 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:01:11,845 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:11,845 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:01:11,845 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:11,846 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:01:11,846 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:11,846 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-12 04:01:11,846 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:01:11,847 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:01:11,847 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b to restore RIT 2023-02-12 04:01:11,847 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174501847 2023-02-12 04:01:11,847 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:01:11,848 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:01:11,850 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:11,850 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=0dc25aecee603c971d1e12ebe66ec89b, regionState=OPENING, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:01:11,850 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:01:11,851 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174531850 2023-02-12 04:01:11,851 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174591851 2023-02-12 04:01:11,851 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:01:11,851 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=0dc25aecee603c971d1e12ebe66ec89b, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:11,851 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174471851"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174471851"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174471851"}]},"ts":"1676174471851"} 2023-02-12 04:01:11,853 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:01:12,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:12,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0dc25aecee603c971d1e12ebe66ec89b, NAME => 'testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:01:12,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:01:12,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:12,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:12,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:12,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:12,010 INFO [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:12,011 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1 2023-02-12 04:01:12,011 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1 2023-02-12 04:01:12,012 INFO [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0dc25aecee603c971d1e12ebe66ec89b columnFamilyName cf1 2023-02-12 04:01:12,018 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2425e2074a554800a1cb2b054f37c4ee 2023-02-12 04:01:12,023 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2644ad6f0d634d25a1cd45968f4a3f23 2023-02-12 04:01:12,027 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2dfc9429c8e449e481b3bbe13ce0f1ae 2023-02-12 04:01:12,032 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/587ab1e026244110a0337602d96b400b 2023-02-12 04:01:12,036 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/b226d99488c3445cb25f1f8c4804e97d 2023-02-12 04:01:12,041 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/c944f93481a14a938f74fa827d2794ca 2023-02-12 04:01:12,044 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/d07f9eb8feeb4de381f15cf69fadacfd 2023-02-12 04:01:12,045 INFO [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(310): Store=0dc25aecee603c971d1e12ebe66ec89b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:12,045 INFO [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:12,046 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2 2023-02-12 04:01:12,046 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2 2023-02-12 04:01:12,046 INFO [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0dc25aecee603c971d1e12ebe66ec89b columnFamilyName cf2 2023-02-12 04:01:12,059 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/03c07a9889c54f9699de00002a5cb893 2023-02-12 04:01:12,063 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/1722a28298d54844a883882c3886f26c 2023-02-12 04:01:12,066 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/2c7c1221ab184268b7d7515c67a3a6a0 2023-02-12 04:01:12,070 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/30bc1796b30540cf992d5c27f95594ed 2023-02-12 04:01:12,074 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/984af9a12e924e769e60064568763d89 2023-02-12 04:01:12,079 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/a87311e55dd84b8ca921d2a5fb51c38c 2023-02-12 04:01:12,083 DEBUG [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/eed9fa1e416247c5be591ec1a6ddf0ec 2023-02-12 04:01:12,083 INFO [StoreOpener-0dc25aecee603c971d1e12ebe66ec89b-1] regionserver.HStore(310): Store=0dc25aecee603c971d1e12ebe66ec89b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:12,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:12,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:12,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.splits 2023-02-12 04:01:12,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:01:12,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:12,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 0dc25aecee603c971d1e12ebe66ec89b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60552090, jitterRate=-0.09770354628562927}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:01:12,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 0dc25aecee603c971d1e12ebe66ec89b: Running coprocessor pre-open hook at 1676174472009Writing region info on filesystem at 1676174472009Initializing all the Stores at 1676174472010 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174472010Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174472010Cleaning up temporary data from old regions at 1676174472085 (+75 ms)Cleaning up detritus from prior splits at 1676174472086 (+1 ms)Running coprocessor post-open hooks at 1676174472089 (+3 ms)Region opened successfully at 1676174472089 2023-02-12 04:01:12,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b., pid=48, masterSystemTime=1676174472005 2023-02-12 04:01:12,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:12,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:12,091 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:01:12,091 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:01:12,092 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-12 04:01:12,092 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-12 04:01:12,092 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1912): 0dc25aecee603c971d1e12ebe66ec89b/cf2 is initiating minor compaction (all files) 2023-02-12 04:01:12,092 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1912): 0dc25aecee603c971d1e12ebe66ec89b/cf1 is initiating minor compaction (all files) 2023-02-12 04:01:12,092 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 0dc25aecee603c971d1e12ebe66ec89b/cf2 in testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:12,092 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0dc25aecee603c971d1e12ebe66ec89b/cf1 in testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:12,092 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/b226d99488c3445cb25f1f8c4804e97d, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/587ab1e026244110a0337602d96b400b, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/c944f93481a14a938f74fa827d2794ca, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2644ad6f0d634d25a1cd45968f4a3f23, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/d07f9eb8feeb4de381f15cf69fadacfd, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2dfc9429c8e449e481b3bbe13ce0f1ae, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2425e2074a554800a1cb2b054f37c4ee] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp, totalSize=34.9 K 2023-02-12 04:01:12,092 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/2c7c1221ab184268b7d7515c67a3a6a0, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/a87311e55dd84b8ca921d2a5fb51c38c, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/30bc1796b30540cf992d5c27f95594ed, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/eed9fa1e416247c5be591ec1a6ddf0ec, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/03c07a9889c54f9699de00002a5cb893, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/1722a28298d54844a883882c3886f26c, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/984af9a12e924e769e60064568763d89] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp, totalSize=34.9 K 2023-02-12 04:01:12,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:12,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:12,094 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 2c7c1221ab184268b7d7515c67a3a6a0, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174464564 2023-02-12 04:01:12,094 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting b226d99488c3445cb25f1f8c4804e97d, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174464564 2023-02-12 04:01:12,094 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=0dc25aecee603c971d1e12ebe66ec89b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:12,094 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting a87311e55dd84b8ca921d2a5fb51c38c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174464744 2023-02-12 04:01:12,094 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174472094"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174472094"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174472094"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174472094"}]},"ts":"1676174472094"} 2023-02-12 04:01:12,094 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 587ab1e026244110a0337602d96b400b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174464744 2023-02-12 04:01:12,094 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 30bc1796b30540cf992d5c27f95594ed, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174464929 2023-02-12 04:01:12,095 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting c944f93481a14a938f74fa827d2794ca, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174464929 2023-02-12 04:01:12,095 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting eed9fa1e416247c5be591ec1a6ddf0ec, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174465127 2023-02-12 04:01:12,095 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 2644ad6f0d634d25a1cd45968f4a3f23, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174465127 2023-02-12 04:01:12,095 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 03c07a9889c54f9699de00002a5cb893, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174465717 2023-02-12 04:01:12,095 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting d07f9eb8feeb4de381f15cf69fadacfd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174465717 2023-02-12 04:01:12,096 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 1722a28298d54844a883882c3886f26c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174465898 2023-02-12 04:01:12,096 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 2dfc9429c8e449e481b3bbe13ce0f1ae, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174465898 2023-02-12 04:01:12,096 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 984af9a12e924e769e60064568763d89, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676174466079 2023-02-12 04:01:12,096 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 2425e2074a554800a1cb2b054f37c4ee, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676174466079 2023-02-12 04:01:12,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-12 04:01:12,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 in 243 msec 2023-02-12 04:01:12,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN in 1.5630 sec 2023-02-12 04:01:12,102 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-12 04:01:12,104 DEBUG [Listener at localhost.localdomain/35207] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-12 04:01:12,106 INFO [Listener at localhost.localdomain/35207] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-12 04:01:12,113 INFO [Listener at localhost.localdomain/35207] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-12 04:01:12,114 INFO [Listener at localhost.localdomain/35207] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-12 04:01:12,114 INFO [Listener at localhost.localdomain/35207] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-12 04:01:12,120 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] throttle.PressureAwareThroughputController(145): 0dc25aecee603c971d1e12ebe66ec89b#cf2#compaction#50 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:12,128 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0dc25aecee603c971d1e12ebe66ec89b#cf1#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:12,147 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf2/596af81da1f14f1a84af86a3744f8db7 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/596af81da1f14f1a84af86a3744f8db7 2023-02-12 04:01:12,154 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0dc25aecee603c971d1e12ebe66ec89b/cf2 of 0dc25aecee603c971d1e12ebe66ec89b into 596af81da1f14f1a84af86a3744f8db7(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:12,154 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0dc25aecee603c971d1e12ebe66ec89b: 2023-02-12 04:01:12,154 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/.tmp/cf1/f4608296a98941d9bb2808138bd40e71 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/f4608296a98941d9bb2808138bd40e71 2023-02-12 04:01:12,154 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b., storeName=0dc25aecee603c971d1e12ebe66ec89b/cf2, priority=9, startTime=1676174472091; duration=0sec 2023-02-12 04:01:12,156 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:12,164 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0dc25aecee603c971d1e12ebe66ec89b/cf1 of 0dc25aecee603c971d1e12ebe66ec89b into f4608296a98941d9bb2808138bd40e71(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:12,164 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0dc25aecee603c971d1e12ebe66ec89b: 2023-02-12 04:01:12,164 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b., storeName=0dc25aecee603c971d1e12ebe66ec89b/cf1, priority=9, startTime=1676174472090; duration=0sec 2023-02-12 04:01:12,167 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:12,175 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:01:12,177 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-12 04:01:12,177 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testRollbackAndDoubleExecution 2023-02-12 04:01:12,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:12,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-12 04:01:12,181 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174472181"}]},"ts":"1676174472181"} 2023-02-12 04:01:12,182 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-12 04:01:12,184 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-12 04:01:12,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN}] 2023-02-12 04:01:12,185 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN 2023-02-12 04:01:12,186 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:12,186 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174472185"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174472185"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174472185"}]},"ts":"1676174472185"} 2023-02-12 04:01:12,187 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:01:12,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-12 04:01:12,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:12,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 0dc25aecee603c971d1e12ebe66ec89b, disabling compactions & flushes 2023-02-12 04:01:12,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:12,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:12,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. after waiting 0 ms 2023-02-12 04:01:12,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:12,340 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/b226d99488c3445cb25f1f8c4804e97d, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/587ab1e026244110a0337602d96b400b, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/c944f93481a14a938f74fa827d2794ca, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2644ad6f0d634d25a1cd45968f4a3f23, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/d07f9eb8feeb4de381f15cf69fadacfd, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2dfc9429c8e449e481b3bbe13ce0f1ae, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2425e2074a554800a1cb2b054f37c4ee] to archive 2023-02-12 04:01:12,341 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:01:12,342 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/b226d99488c3445cb25f1f8c4804e97d to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/b226d99488c3445cb25f1f8c4804e97d 2023-02-12 04:01:12,344 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/587ab1e026244110a0337602d96b400b to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/587ab1e026244110a0337602d96b400b 2023-02-12 04:01:12,345 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/c944f93481a14a938f74fa827d2794ca to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/c944f93481a14a938f74fa827d2794ca 2023-02-12 04:01:12,347 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2644ad6f0d634d25a1cd45968f4a3f23 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2644ad6f0d634d25a1cd45968f4a3f23 2023-02-12 04:01:12,348 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/d07f9eb8feeb4de381f15cf69fadacfd to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/d07f9eb8feeb4de381f15cf69fadacfd 2023-02-12 04:01:12,349 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2dfc9429c8e449e481b3bbe13ce0f1ae to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2dfc9429c8e449e481b3bbe13ce0f1ae 2023-02-12 04:01:12,351 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2425e2074a554800a1cb2b054f37c4ee to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/2425e2074a554800a1cb2b054f37c4ee 2023-02-12 04:01:12,355 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/2c7c1221ab184268b7d7515c67a3a6a0, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/a87311e55dd84b8ca921d2a5fb51c38c, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/30bc1796b30540cf992d5c27f95594ed, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/eed9fa1e416247c5be591ec1a6ddf0ec, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/03c07a9889c54f9699de00002a5cb893, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/1722a28298d54844a883882c3886f26c, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/984af9a12e924e769e60064568763d89] to archive 2023-02-12 04:01:12,356 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:01:12,358 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/2c7c1221ab184268b7d7515c67a3a6a0 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/2c7c1221ab184268b7d7515c67a3a6a0 2023-02-12 04:01:12,359 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/a87311e55dd84b8ca921d2a5fb51c38c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/a87311e55dd84b8ca921d2a5fb51c38c 2023-02-12 04:01:12,361 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/30bc1796b30540cf992d5c27f95594ed to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/30bc1796b30540cf992d5c27f95594ed 2023-02-12 04:01:12,362 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/eed9fa1e416247c5be591ec1a6ddf0ec to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/eed9fa1e416247c5be591ec1a6ddf0ec 2023-02-12 04:01:12,363 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/03c07a9889c54f9699de00002a5cb893 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/03c07a9889c54f9699de00002a5cb893 2023-02-12 04:01:12,364 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/1722a28298d54844a883882c3886f26c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/1722a28298d54844a883882c3886f26c 2023-02-12 04:01:12,366 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/984af9a12e924e769e60064568763d89 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/984af9a12e924e769e60064568763d89 2023-02-12 04:01:12,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-12 04:01:12,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:01:12,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b. 2023-02-12 04:01:12,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 0dc25aecee603c971d1e12ebe66ec89b: Waiting for close lock at 1676174472339Running coprocessor pre-close hooks at 1676174472339Disabling compacts and flushes for region at 1676174472339Disabling writes for close at 1676174472339Writing region close event to WAL at 1676174472367 (+28 ms)Running coprocessor post-close hooks at 1676174472370 (+3 ms)Closed at 1676174472370 2023-02-12 04:01:12,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:12,372 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=0dc25aecee603c971d1e12ebe66ec89b, regionState=CLOSED 2023-02-12 04:01:12,372 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174472372"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174472372"}]},"ts":"1676174472372"} 2023-02-12 04:01:12,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-12 04:01:12,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 0dc25aecee603c971d1e12ebe66ec89b, server=jenkins-hbase20.apache.org,43977,1676174449184 in 186 msec 2023-02-12 04:01:12,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-12 04:01:12,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, UNASSIGN in 191 msec 2023-02-12 04:01:12,378 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174472377"}]},"ts":"1676174472377"} 2023-02-12 04:01:12,379 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-12 04:01:12,380 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-12 04:01:12,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 203 msec 2023-02-12 04:01:12,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-12 04:01:12,484 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-12 04:01:12,484 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testRollbackAndDoubleExecution 2023-02-12 04:01:12,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:12,487 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:12,488 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:12,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-12 04:01:12,491 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:12,493 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/recovered.edits] 2023-02-12 04:01:12,495 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/f4608296a98941d9bb2808138bd40e71 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf1/f4608296a98941d9bb2808138bd40e71 2023-02-12 04:01:12,498 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/596af81da1f14f1a84af86a3744f8db7 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/cf2/596af81da1f14f1a84af86a3744f8db7 2023-02-12 04:01:12,500 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/recovered.edits/90.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b/recovered.edits/90.seqid 2023-02-12 04:01:12,501 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackAndDoubleExecution/0dc25aecee603c971d1e12ebe66ec89b 2023-02-12 04:01:12,501 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-12 04:01:12,505 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:12,506 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-12 04:01:12,508 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-12 04:01:12,509 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:12,509 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-12 04:01:12,509 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174472509"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:12,511 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-12 04:01:12,511 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0dc25aecee603c971d1e12ebe66ec89b, NAME => 'testRollbackAndDoubleExecution,,1676174464060.0dc25aecee603c971d1e12ebe66ec89b.', STARTKEY => '', ENDKEY => ''}] 2023-02-12 04:01:12,511 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-12 04:01:12,511 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676174472511"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:12,513 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-12 04:01:12,515 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:01:12,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 30 msec 2023-02-12 04:01:12,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-12 04:01:12,590 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-12 04:01:12,603 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=392 (was 395), OpenFileDescriptor=849 (was 861), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=196 (was 196), ProcessCount=165 (was 165), AvailableMemoryMB=4138 (was 4153) 2023-02-12 04:01:12,617 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=392, OpenFileDescriptor=849, MaxFileDescriptor=60000, SystemLoadAverage=196, ProcessCount=165, AvailableMemoryMB=4137 2023-02-12 04:01:12,617 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:01:12,620 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-12 04:01:12,620 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-12 04:01:12,622 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:01:12,623 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 04:01:12,624 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 04:01:12,626 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:12,626 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc empty. 2023-02-12 04:01:12,627 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:12,627 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-12 04:01:12,635 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-12 04:01:12,636 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => ea775fa04e6f5b6170f1eb87d6c0efbc, NAME => 'testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp 2023-02-12 04:01:12,643 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:12,643 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing ea775fa04e6f5b6170f1eb87d6c0efbc, disabling compactions & flushes 2023-02-12 04:01:12,643 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc. 2023-02-12 04:01:12,644 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc. 2023-02-12 04:01:12,644 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc. after waiting 0 ms 2023-02-12 04:01:12,644 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc. 2023-02-12 04:01:12,644 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc. 2023-02-12 04:01:12,644 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for ea775fa04e6f5b6170f1eb87d6c0efbc: Waiting for close lock at 1676174472643Disabling compacts and flushes for region at 1676174472643Disabling writes for close at 1676174472644 (+1 ms)Writing region close event to WAL at 1676174472644Closed at 1676174472644 2023-02-12 04:01:12,651 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 04:01:12,652 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676174472651"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174472651"}]},"ts":"1676174472651"} 2023-02-12 04:01:12,653 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 04:01:12,654 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 04:01:12,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174472655"}]},"ts":"1676174472655"} 2023-02-12 04:01:12,656 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-12 04:01:12,658 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-12 04:01:12,659 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-12 04:01:12,659 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-12 04:01:12,659 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-12 04:01:12,659 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-12 04:01:12,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ea775fa04e6f5b6170f1eb87d6c0efbc, ASSIGN}] 2023-02-12 04:01:12,661 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ea775fa04e6f5b6170f1eb87d6c0efbc, ASSIGN 2023-02-12 04:01:12,662 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ea775fa04e6f5b6170f1eb87d6c0efbc, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,40151,1676174449139; forceNewPlan=false, retain=false 2023-02-12 04:01:12,812 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-12 04:01:12,813 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=ea775fa04e6f5b6170f1eb87d6c0efbc, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:12,813 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676174472813"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174472813"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174472813"}]},"ts":"1676174472813"} 2023-02-12 04:01:12,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure ea775fa04e6f5b6170f1eb87d6c0efbc, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:12,967 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:12,967 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-12 04:01:12,970 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:34246, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-12 04:01:12,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc. 2023-02-12 04:01:12,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ea775fa04e6f5b6170f1eb87d6c0efbc, NAME => 'testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:01:12,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:01:12,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:12,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:12,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:12,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:12,979 INFO [StoreOpener-ea775fa04e6f5b6170f1eb87d6c0efbc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:12,981 DEBUG [StoreOpener-ea775fa04e6f5b6170f1eb87d6c0efbc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc/cf1 2023-02-12 04:01:12,981 DEBUG [StoreOpener-ea775fa04e6f5b6170f1eb87d6c0efbc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc/cf1 2023-02-12 04:01:12,981 INFO [StoreOpener-ea775fa04e6f5b6170f1eb87d6c0efbc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea775fa04e6f5b6170f1eb87d6c0efbc columnFamilyName cf1 2023-02-12 04:01:12,982 INFO [StoreOpener-ea775fa04e6f5b6170f1eb87d6c0efbc-1] regionserver.HStore(310): Store=ea775fa04e6f5b6170f1eb87d6c0efbc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:12,982 INFO [StoreOpener-ea775fa04e6f5b6170f1eb87d6c0efbc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:12,984 DEBUG [StoreOpener-ea775fa04e6f5b6170f1eb87d6c0efbc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc/cf2 2023-02-12 04:01:12,984 DEBUG [StoreOpener-ea775fa04e6f5b6170f1eb87d6c0efbc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc/cf2 2023-02-12 04:01:12,984 INFO [StoreOpener-ea775fa04e6f5b6170f1eb87d6c0efbc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea775fa04e6f5b6170f1eb87d6c0efbc columnFamilyName cf2 2023-02-12 04:01:12,985 INFO [StoreOpener-ea775fa04e6f5b6170f1eb87d6c0efbc-1] regionserver.HStore(310): Store=ea775fa04e6f5b6170f1eb87d6c0efbc/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:12,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:12,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:12,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:01:12,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:12,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 04:01:12,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened ea775fa04e6f5b6170f1eb87d6c0efbc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64681627, jitterRate=-0.036168649792671204}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:01:12,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for ea775fa04e6f5b6170f1eb87d6c0efbc: Running coprocessor pre-open hook at 1676174472978Writing region info on filesystem at 1676174472978Initializing all the Stores at 1676174472979 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174472979Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174472979Cleaning up temporary data from old regions at 1676174472987 (+8 ms)Cleaning up detritus from prior splits at 1676174472987Running coprocessor post-open hooks at 1676174472993 (+6 ms)Region opened successfully at 1676174472993 2023-02-12 04:01:12,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc., pid=55, masterSystemTime=1676174472967 2023-02-12 04:01:12,998 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc. 2023-02-12 04:01:12,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc. 2023-02-12 04:01:13,000 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=ea775fa04e6f5b6170f1eb87d6c0efbc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:13,000 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676174473000"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174473000"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174473000"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174473000"}]},"ts":"1676174473000"} 2023-02-12 04:01:13,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-12 04:01:13,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure ea775fa04e6f5b6170f1eb87d6c0efbc, server=jenkins-hbase20.apache.org,40151,1676174449139 in 187 msec 2023-02-12 04:01:13,007 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-12 04:01:13,007 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ea775fa04e6f5b6170f1eb87d6c0efbc, ASSIGN in 346 msec 2023-02-12 04:01:13,008 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 04:01:13,008 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174473008"}]},"ts":"1676174473008"} 2023-02-12 04:01:13,010 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-12 04:01:13,012 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 04:01:13,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 391 msec 2023-02-12 04:01:13,123 DEBUG [Listener at localhost.localdomain/35207] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:13,124 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:01:13,125 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:01:13,125 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:01:13,125 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:01:13,127 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ea775fa04e6f5b6170f1eb87d6c0efbc, UNASSIGN}] 2023-02-12 04:01:13,128 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ea775fa04e6f5b6170f1eb87d6c0efbc, UNASSIGN 2023-02-12 04:01:13,129 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=ea775fa04e6f5b6170f1eb87d6c0efbc, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:13,129 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676174473128"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174473128"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174473128"}]},"ts":"1676174473128"} 2023-02-12 04:01:13,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure ea775fa04e6f5b6170f1eb87d6c0efbc, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:13,283 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:13,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing ea775fa04e6f5b6170f1eb87d6c0efbc, disabling compactions & flushes 2023-02-12 04:01:13,284 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc. 2023-02-12 04:01:13,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc. 2023-02-12 04:01:13,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc. after waiting 0 ms 2023-02-12 04:01:13,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc. 2023-02-12 04:01:13,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-12 04:01:13,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:01:13,288 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc. 2023-02-12 04:01:13,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for ea775fa04e6f5b6170f1eb87d6c0efbc: Waiting for close lock at 1676174473283Running coprocessor pre-close hooks at 1676174473283Disabling compacts and flushes for region at 1676174473283Disabling writes for close at 1676174473284 (+1 ms)Writing region close event to WAL at 1676174473284Running coprocessor post-close hooks at 1676174473288 (+4 ms)Closed at 1676174473288 2023-02-12 04:01:13,289 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:13,290 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=ea775fa04e6f5b6170f1eb87d6c0efbc, regionState=CLOSED 2023-02-12 04:01:13,290 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676174473290"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174473290"}]},"ts":"1676174473290"} 2023-02-12 04:01:13,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-12 04:01:13,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure ea775fa04e6f5b6170f1eb87d6c0efbc, server=jenkins-hbase20.apache.org,40151,1676174449139 in 161 msec 2023-02-12 04:01:13,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-12 04:01:13,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ea775fa04e6f5b6170f1eb87d6c0efbc, UNASSIGN in 167 msec 2023-02-12 04:01:13,326 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-12 04:01:13,328 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-12 04:01:13,330 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676174473329"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676174473329"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676174473329"}]},"ts":"1676174473329"} 2023-02-12 04:01:13,330 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174473329"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174473329"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174473329"}]},"ts":"1676174473329"} 2023-02-12 04:01:13,330 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174473329"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174473329"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174473329"}]},"ts":"1676174473329"} 2023-02-12 04:01:13,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6d457f69a40d1d02a5962902562a788, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ced688347822f331e4682308ef38f26, ASSIGN}] 2023-02-12 04:01:13,335 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ced688347822f331e4682308ef38f26, ASSIGN 2023-02-12 04:01:13,335 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ced688347822f331e4682308ef38f26, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,40151,1676174449139; forceNewPlan=false, retain=false 2023-02-12 04:01:13,336 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6d457f69a40d1d02a5962902562a788, ASSIGN 2023-02-12 04:01:13,337 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6d457f69a40d1d02a5962902562a788, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,40151,1676174449139; forceNewPlan=false, retain=false 2023-02-12 04:01:13,486 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-12 04:01:13,487 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=d6d457f69a40d1d02a5962902562a788, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:13,487 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174473487"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174473487"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174473487"}]},"ts":"1676174473487"} 2023-02-12 04:01:13,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure d6d457f69a40d1d02a5962902562a788, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:13,492 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=6ced688347822f331e4682308ef38f26, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:13,492 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174473491"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174473491"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174473491"}]},"ts":"1676174473491"} 2023-02-12 04:01:13,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 6ced688347822f331e4682308ef38f26, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:13,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788. 2023-02-12 04:01:13,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d6d457f69a40d1d02a5962902562a788, NAME => 'testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788.', STARTKEY => '', ENDKEY => '41'} 2023-02-12 04:01:13,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:01:13,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile d6d457f69a40d1d02a5962902562a788 2023-02-12 04:01:13,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:13,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for d6d457f69a40d1d02a5962902562a788 2023-02-12 04:01:13,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for d6d457f69a40d1d02a5962902562a788 2023-02-12 04:01:13,646 INFO [StoreOpener-d6d457f69a40d1d02a5962902562a788-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d6d457f69a40d1d02a5962902562a788 2023-02-12 04:01:13,647 DEBUG [StoreOpener-d6d457f69a40d1d02a5962902562a788-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788/cf1 2023-02-12 04:01:13,647 DEBUG [StoreOpener-d6d457f69a40d1d02a5962902562a788-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788/cf1 2023-02-12 04:01:13,647 INFO [StoreOpener-d6d457f69a40d1d02a5962902562a788-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d6d457f69a40d1d02a5962902562a788 columnFamilyName cf1 2023-02-12 04:01:13,648 INFO [StoreOpener-d6d457f69a40d1d02a5962902562a788-1] regionserver.HStore(310): Store=d6d457f69a40d1d02a5962902562a788/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:13,648 INFO [StoreOpener-d6d457f69a40d1d02a5962902562a788-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d6d457f69a40d1d02a5962902562a788 2023-02-12 04:01:13,649 DEBUG [StoreOpener-d6d457f69a40d1d02a5962902562a788-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788/cf2 2023-02-12 04:01:13,650 DEBUG [StoreOpener-d6d457f69a40d1d02a5962902562a788-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788/cf2 2023-02-12 04:01:13,650 INFO [StoreOpener-d6d457f69a40d1d02a5962902562a788-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d6d457f69a40d1d02a5962902562a788 columnFamilyName cf2 2023-02-12 04:01:13,650 INFO [StoreOpener-d6d457f69a40d1d02a5962902562a788-1] regionserver.HStore(310): Store=d6d457f69a40d1d02a5962902562a788/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:13,651 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788 2023-02-12 04:01:13,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788 2023-02-12 04:01:13,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:01:13,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for d6d457f69a40d1d02a5962902562a788 2023-02-12 04:01:13,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened d6d457f69a40d1d02a5962902562a788; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59609337, jitterRate=-0.11175166070461273}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:01:13,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for d6d457f69a40d1d02a5962902562a788: Running coprocessor pre-open hook at 1676174473644Writing region info on filesystem at 1676174473644Initializing all the Stores at 1676174473645 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174473645Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174473645Cleaning up temporary data from old regions at 1676174473653 (+8 ms)Cleaning up detritus from prior splits at 1676174473654 (+1 ms)Running coprocessor post-open hooks at 1676174473656 (+2 ms)Region opened successfully at 1676174473656 2023-02-12 04:01:13,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788., pid=61, masterSystemTime=1676174473641 2023-02-12 04:01:13,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788. 2023-02-12 04:01:13,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788. 2023-02-12 04:01:13,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26. 2023-02-12 04:01:13,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6ced688347822f331e4682308ef38f26, NAME => 'testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26.', STARTKEY => '41', ENDKEY => ''} 2023-02-12 04:01:13,658 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=d6d457f69a40d1d02a5962902562a788, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:13,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:01:13,659 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174473658"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174473658"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174473658"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174473658"}]},"ts":"1676174473658"} 2023-02-12 04:01:13,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 6ced688347822f331e4682308ef38f26 2023-02-12 04:01:13,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:13,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 6ced688347822f331e4682308ef38f26 2023-02-12 04:01:13,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 6ced688347822f331e4682308ef38f26 2023-02-12 04:01:13,660 INFO [StoreOpener-6ced688347822f331e4682308ef38f26-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6ced688347822f331e4682308ef38f26 2023-02-12 04:01:13,662 DEBUG [StoreOpener-6ced688347822f331e4682308ef38f26-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26/cf1 2023-02-12 04:01:13,662 DEBUG [StoreOpener-6ced688347822f331e4682308ef38f26-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26/cf1 2023-02-12 04:01:13,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-12 04:01:13,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure d6d457f69a40d1d02a5962902562a788, server=jenkins-hbase20.apache.org,40151,1676174449139 in 171 msec 2023-02-12 04:01:13,662 INFO [StoreOpener-6ced688347822f331e4682308ef38f26-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ced688347822f331e4682308ef38f26 columnFamilyName cf1 2023-02-12 04:01:13,663 INFO [StoreOpener-6ced688347822f331e4682308ef38f26-1] regionserver.HStore(310): Store=6ced688347822f331e4682308ef38f26/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:13,663 INFO [StoreOpener-6ced688347822f331e4682308ef38f26-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6ced688347822f331e4682308ef38f26 2023-02-12 04:01:13,663 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6d457f69a40d1d02a5962902562a788, ASSIGN in 328 msec 2023-02-12 04:01:13,664 DEBUG [StoreOpener-6ced688347822f331e4682308ef38f26-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26/cf2 2023-02-12 04:01:13,664 DEBUG [StoreOpener-6ced688347822f331e4682308ef38f26-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26/cf2 2023-02-12 04:01:13,665 INFO [StoreOpener-6ced688347822f331e4682308ef38f26-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ced688347822f331e4682308ef38f26 columnFamilyName cf2 2023-02-12 04:01:13,665 INFO [StoreOpener-6ced688347822f331e4682308ef38f26-1] regionserver.HStore(310): Store=6ced688347822f331e4682308ef38f26/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:13,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26 2023-02-12 04:01:13,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26 2023-02-12 04:01:13,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:01:13,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 6ced688347822f331e4682308ef38f26 2023-02-12 04:01:13,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 6ced688347822f331e4682308ef38f26; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62675654, jitterRate=-0.06605997681617737}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:01:13,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 6ced688347822f331e4682308ef38f26: Running coprocessor pre-open hook at 1676174473659Writing region info on filesystem at 1676174473659Initializing all the Stores at 1676174473660 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174473660Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174473660Cleaning up temporary data from old regions at 1676174473668 (+8 ms)Cleaning up detritus from prior splits at 1676174473668Running coprocessor post-open hooks at 1676174473672 (+4 ms)Region opened successfully at 1676174473672 2023-02-12 04:01:13,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26., pid=62, masterSystemTime=1676174473641 2023-02-12 04:01:13,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26. 2023-02-12 04:01:13,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26. 2023-02-12 04:01:13,674 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=6ced688347822f331e4682308ef38f26, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:13,675 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174473674"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174473674"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174473674"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174473674"}]},"ts":"1676174473674"} 2023-02-12 04:01:13,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-12 04:01:13,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 6ced688347822f331e4682308ef38f26, server=jenkins-hbase20.apache.org,40151,1676174449139 in 183 msec 2023-02-12 04:01:13,679 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-02-12 04:01:13,679 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ced688347822f331e4682308ef38f26, ASSIGN in 344 msec 2023-02-12 04:01:13,680 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 in 556 msec 2023-02-12 04:01:13,878 DEBUG [hconnection-0x19f60b37-shared-pool-11] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 04:01:13,881 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:53720, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 04:01:13,883 DEBUG [Listener at localhost.localdomain/35207] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-12 04:01:13,886 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:01:13,887 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-12 04:01:13,887 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testSplitTableRegionNoStoreFile 2023-02-12 04:01:13,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:01:13,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-12 04:01:13,891 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174473890"}]},"ts":"1676174473890"} 2023-02-12 04:01:13,892 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-12 04:01:13,893 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-12 04:01:13,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6d457f69a40d1d02a5962902562a788, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ced688347822f331e4682308ef38f26, UNASSIGN}] 2023-02-12 04:01:13,894 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ced688347822f331e4682308ef38f26, UNASSIGN 2023-02-12 04:01:13,895 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=6ced688347822f331e4682308ef38f26, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:13,895 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174473895"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174473895"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174473895"}]},"ts":"1676174473895"} 2023-02-12 04:01:13,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 6ced688347822f331e4682308ef38f26, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:13,899 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6d457f69a40d1d02a5962902562a788, UNASSIGN 2023-02-12 04:01:13,899 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=d6d457f69a40d1d02a5962902562a788, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:13,900 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174473899"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174473899"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174473899"}]},"ts":"1676174473899"} 2023-02-12 04:01:13,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure d6d457f69a40d1d02a5962902562a788, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:13,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-12 04:01:14,048 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close d6d457f69a40d1d02a5962902562a788 2023-02-12 04:01:14,049 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing d6d457f69a40d1d02a5962902562a788, disabling compactions & flushes 2023-02-12 04:01:14,050 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788. 2023-02-12 04:01:14,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788. 2023-02-12 04:01:14,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788. after waiting 0 ms 2023-02-12 04:01:14,050 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788. 2023-02-12 04:01:14,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-12 04:01:14,055 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:01:14,055 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788. 2023-02-12 04:01:14,055 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for d6d457f69a40d1d02a5962902562a788: Waiting for close lock at 1676174474049Running coprocessor pre-close hooks at 1676174474049Disabling compacts and flushes for region at 1676174474049Disabling writes for close at 1676174474050 (+1 ms)Writing region close event to WAL at 1676174474050Running coprocessor post-close hooks at 1676174474055 (+5 ms)Closed at 1676174474055 2023-02-12 04:01:14,057 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed d6d457f69a40d1d02a5962902562a788 2023-02-12 04:01:14,057 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 6ced688347822f331e4682308ef38f26 2023-02-12 04:01:14,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 6ced688347822f331e4682308ef38f26, disabling compactions & flushes 2023-02-12 04:01:14,058 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26. 2023-02-12 04:01:14,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26. 2023-02-12 04:01:14,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26. after waiting 0 ms 2023-02-12 04:01:14,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26. 2023-02-12 04:01:14,058 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=d6d457f69a40d1d02a5962902562a788, regionState=CLOSED 2023-02-12 04:01:14,058 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174474058"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174474058"}]},"ts":"1676174474058"} 2023-02-12 04:01:14,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-12 04:01:14,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:01:14,062 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26. 2023-02-12 04:01:14,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 6ced688347822f331e4682308ef38f26: Waiting for close lock at 1676174474057Running coprocessor pre-close hooks at 1676174474057Disabling compacts and flushes for region at 1676174474057Disabling writes for close at 1676174474058 (+1 ms)Writing region close event to WAL at 1676174474058Running coprocessor post-close hooks at 1676174474062 (+4 ms)Closed at 1676174474062 2023-02-12 04:01:14,068 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 6ced688347822f331e4682308ef38f26 2023-02-12 04:01:14,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-12 04:01:14,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure d6d457f69a40d1d02a5962902562a788, server=jenkins-hbase20.apache.org,40151,1676174449139 in 166 msec 2023-02-12 04:01:14,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d6d457f69a40d1d02a5962902562a788, UNASSIGN in 175 msec 2023-02-12 04:01:14,071 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=6ced688347822f331e4682308ef38f26, regionState=CLOSED 2023-02-12 04:01:14,071 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174474071"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174474071"}]},"ts":"1676174474071"} 2023-02-12 04:01:14,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-12 04:01:14,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 6ced688347822f331e4682308ef38f26, server=jenkins-hbase20.apache.org,40151,1676174449139 in 177 msec 2023-02-12 04:01:14,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-02-12 04:01:14,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ced688347822f331e4682308ef38f26, UNASSIGN in 180 msec 2023-02-12 04:01:14,077 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174474077"}]},"ts":"1676174474077"} 2023-02-12 04:01:14,079 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-12 04:01:14,080 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-12 04:01:14,082 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 193 msec 2023-02-12 04:01:14,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-12 04:01:14,193 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-12 04:01:14,194 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testSplitTableRegionNoStoreFile 2023-02-12 04:01:14,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:01:14,197 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:01:14,198 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:01:14,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-12 04:01:14,202 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26 2023-02-12 04:01:14,202 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:14,202 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788 2023-02-12 04:01:14,204 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788/recovered.edits] 2023-02-12 04:01:14,205 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26/recovered.edits] 2023-02-12 04:01:14,205 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc/recovered.edits] 2023-02-12 04:01:14,211 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26/recovered.edits/7.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26/recovered.edits/7.seqid 2023-02-12 04:01:14,211 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788/recovered.edits/7.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788/recovered.edits/7.seqid 2023-02-12 04:01:14,211 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc/recovered.edits/4.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc/recovered.edits/4.seqid 2023-02-12 04:01:14,212 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/6ced688347822f331e4682308ef38f26 2023-02-12 04:01:14,212 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/d6d457f69a40d1d02a5962902562a788 2023-02-12 04:01:14,212 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionNoStoreFile/ea775fa04e6f5b6170f1eb87d6c0efbc 2023-02-12 04:01:14,212 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-12 04:01:14,215 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:01:14,217 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-12 04:01:14,219 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-12 04:01:14,221 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:01:14,221 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-12 04:01:14,221 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174474221"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:14,221 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174474221"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:14,221 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174474221"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:14,223 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-12 04:01:14,223 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ea775fa04e6f5b6170f1eb87d6c0efbc, NAME => 'testSplitTableRegionNoStoreFile,,1676174472621.ea775fa04e6f5b6170f1eb87d6c0efbc.', STARTKEY => '', ENDKEY => ''}, {ENCODED => d6d457f69a40d1d02a5962902562a788, NAME => 'testSplitTableRegionNoStoreFile,,1676174473123.d6d457f69a40d1d02a5962902562a788.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 6ced688347822f331e4682308ef38f26, NAME => 'testSplitTableRegionNoStoreFile,41,1676174473123.6ced688347822f331e4682308ef38f26.', STARTKEY => '41', ENDKEY => ''}] 2023-02-12 04:01:14,223 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-12 04:01:14,224 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676174474223"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:14,225 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-12 04:01:14,227 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:01:14,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 33 msec 2023-02-12 04:01:14,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-12 04:01:14,299 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-12 04:01:14,315 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=402 (was 392) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-91922391_15 at /127.0.0.1:36372 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-91922391_15 at /127.0.0.1:49390 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=855 (was 849) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=197 (was 196) - SystemLoadAverage LEAK? -, ProcessCount=165 (was 165), AvailableMemoryMB=4134 (was 4137) 2023-02-12 04:01:14,331 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=402, OpenFileDescriptor=855, MaxFileDescriptor=60000, SystemLoadAverage=197, ProcessCount=165, AvailableMemoryMB=4133 2023-02-12 04:01:14,331 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:01:14,334 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-12 04:01:14,334 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-12 04:01:14,335 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,338 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 04:01:14,339 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 04:01:14,340 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:14,341 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57 empty. 2023-02-12 04:01:14,341 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:14,341 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-12 04:01:14,362 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-12 04:01:14,364 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 95c25e17def1a4a4a028fa2b6f6b5c57, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp 2023-02-12 04:01:14,372 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:14,373 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 95c25e17def1a4a4a028fa2b6f6b5c57, disabling compactions & flushes 2023-02-12 04:01:14,373 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:14,373 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:14,373 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. after waiting 0 ms 2023-02-12 04:01:14,373 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:14,373 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:14,373 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 95c25e17def1a4a4a028fa2b6f6b5c57: Waiting for close lock at 1676174474373Disabling compacts and flushes for region at 1676174474373Disabling writes for close at 1676174474373Writing region close event to WAL at 1676174474373Closed at 1676174474373 2023-02-12 04:01:14,376 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 04:01:14,377 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676174474377"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174474377"}]},"ts":"1676174474377"} 2023-02-12 04:01:14,378 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 04:01:14,379 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 04:01:14,379 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174474379"}]},"ts":"1676174474379"} 2023-02-12 04:01:14,381 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-12 04:01:14,383 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-12 04:01:14,384 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-12 04:01:14,384 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-12 04:01:14,384 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-12 04:01:14,384 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-12 04:01:14,384 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=95c25e17def1a4a4a028fa2b6f6b5c57, ASSIGN}] 2023-02-12 04:01:14,386 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=95c25e17def1a4a4a028fa2b6f6b5c57, ASSIGN 2023-02-12 04:01:14,386 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=95c25e17def1a4a4a028fa2b6f6b5c57, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,40151,1676174449139; forceNewPlan=false, retain=false 2023-02-12 04:01:14,537 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-12 04:01:14,538 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=95c25e17def1a4a4a028fa2b6f6b5c57, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,538 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676174474538"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174474538"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174474538"}]},"ts":"1676174474538"} 2023-02-12 04:01:14,540 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 95c25e17def1a4a4a028fa2b6f6b5c57, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:14,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:14,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 95c25e17def1a4a4a028fa2b6f6b5c57, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:01:14,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:01:14,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:14,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:14,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:14,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:14,696 INFO [StoreOpener-95c25e17def1a4a4a028fa2b6f6b5c57-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:14,697 DEBUG [StoreOpener-95c25e17def1a4a4a028fa2b6f6b5c57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1 2023-02-12 04:01:14,697 DEBUG [StoreOpener-95c25e17def1a4a4a028fa2b6f6b5c57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1 2023-02-12 04:01:14,698 INFO [StoreOpener-95c25e17def1a4a4a028fa2b6f6b5c57-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 95c25e17def1a4a4a028fa2b6f6b5c57 columnFamilyName cf1 2023-02-12 04:01:14,698 INFO [StoreOpener-95c25e17def1a4a4a028fa2b6f6b5c57-1] regionserver.HStore(310): Store=95c25e17def1a4a4a028fa2b6f6b5c57/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:14,699 INFO [StoreOpener-95c25e17def1a4a4a028fa2b6f6b5c57-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:14,700 DEBUG [StoreOpener-95c25e17def1a4a4a028fa2b6f6b5c57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2 2023-02-12 04:01:14,700 DEBUG [StoreOpener-95c25e17def1a4a4a028fa2b6f6b5c57-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2 2023-02-12 04:01:14,700 INFO [StoreOpener-95c25e17def1a4a4a028fa2b6f6b5c57-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 95c25e17def1a4a4a028fa2b6f6b5c57 columnFamilyName cf2 2023-02-12 04:01:14,701 INFO [StoreOpener-95c25e17def1a4a4a028fa2b6f6b5c57-1] regionserver.HStore(310): Store=95c25e17def1a4a4a028fa2b6f6b5c57/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:14,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:14,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:14,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:01:14,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:14,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 04:01:14,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 95c25e17def1a4a4a028fa2b6f6b5c57; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69916040, jitterRate=0.04183018207550049}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:01:14,710 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 95c25e17def1a4a4a028fa2b6f6b5c57: Running coprocessor pre-open hook at 1676174474695Writing region info on filesystem at 1676174474695Initializing all the Stores at 1676174474696 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174474696Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174474696Cleaning up temporary data from old regions at 1676174474703 (+7 ms)Cleaning up detritus from prior splits at 1676174474704 (+1 ms)Running coprocessor post-open hooks at 1676174474709 (+5 ms)Region opened successfully at 1676174474709 2023-02-12 04:01:14,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57., pid=71, masterSystemTime=1676174474691 2023-02-12 04:01:14,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:14,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:14,713 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=95c25e17def1a4a4a028fa2b6f6b5c57, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,713 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676174474713"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174474713"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174474713"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174474713"}]},"ts":"1676174474713"} 2023-02-12 04:01:14,717 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-12 04:01:14,717 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 95c25e17def1a4a4a028fa2b6f6b5c57, server=jenkins-hbase20.apache.org,40151,1676174449139 in 174 msec 2023-02-12 04:01:14,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-12 04:01:14,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=95c25e17def1a4a4a028fa2b6f6b5c57, ASSIGN in 333 msec 2023-02-12 04:01:14,720 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 04:01:14,720 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174474720"}]},"ts":"1676174474720"} 2023-02-12 04:01:14,721 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-12 04:01:14,722 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 04:01:14,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 388 msec 2023-02-12 04:01:14,844 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:14,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,845 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:14,845 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:14,845 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-12 04:01:14,845 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-12 04:01:14,846 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,846 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,847 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:14,847 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:14,847 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:14,847 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:14,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:14,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:14,847 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,847 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:14,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:14,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:14,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:14,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,847 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:14,848 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:14,848 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:14,849 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:14,849 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:14,849 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. started... 2023-02-12 04:01:14,850 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 95c25e17def1a4a4a028fa2b6f6b5c57 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-12 04:01:14,850 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,850 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,850 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,850 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:14,851 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,851 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:14,864 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/1b013602f2774d4cb46ddcd5f9afb5db 2023-02-12 04:01:14,878 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/cce1f75ff14d47488df98bf67c7ea161 2023-02-12 04:01:14,882 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/1b013602f2774d4cb46ddcd5f9afb5db as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/1b013602f2774d4cb46ddcd5f9afb5db 2023-02-12 04:01:14,886 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/1b013602f2774d4cb46ddcd5f9afb5db, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:01:14,886 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/cce1f75ff14d47488df98bf67c7ea161 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/cce1f75ff14d47488df98bf67c7ea161 2023-02-12 04:01:14,890 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/cce1f75ff14d47488df98bf67c7ea161, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:01:14,891 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 95c25e17def1a4a4a028fa2b6f6b5c57 in 41ms, sequenceid=6, compaction requested=false 2023-02-12 04:01:14,891 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:14,891 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 95c25e17def1a4a4a028fa2b6f6b5c57: 2023-02-12 04:01:14,891 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:14,891 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:14,891 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:14,891 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,891 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:14,891 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:14,892 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,892 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:14,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:14,893 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,893 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:14,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:14,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:14,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:14,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:14,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:14,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-12 04:01:14,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c63f455[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:14,894 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-12 04:01:14,894 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,895 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,895 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,895 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,895 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:14,895 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,895 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:14,895 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:14,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,895 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:14,895 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:14,895 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:14,895 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:14,895 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:14,896 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:14,896 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:14,896 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:14,896 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,896 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:14,896 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:14,896 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:14,896 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:14,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:14,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:14,896 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:14,896 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:14,897 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:14,897 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:14,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:14,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:14,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:14,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:14,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:14,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:14,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:14,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:01:14,899 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:01:14,899 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-12 04:01:14,899 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:14,899 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:14,899 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,899 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:14,900 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,900 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,900 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,900 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,900 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:14,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,900 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:14,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,900 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:14,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:14,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:14,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:14,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:14,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:14,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:14,901 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:14,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:14,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:14,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:14,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,901 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:14,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,901 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:14,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:14,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:14,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,902 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:14,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:14,903 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:14,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,903 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:14,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:14,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,906 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,906 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:14,906 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:14,906 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:14,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:14,906 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:14,906 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:14,906 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:14,906 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:14,906 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:14,906 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:14,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:14,906 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:14,906 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:14,907 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:14,907 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:14,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:14,907 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:14,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:14,907 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:14,907 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:14,907 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:14,907 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:14,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:14,907 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:14,907 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:14,907 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,007 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:15,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:15,023 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:15,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:01:15,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,025 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:15,025 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,025 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-12 04:01:15,025 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-12 04:01:15,025 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,025 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,026 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,026 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,026 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,026 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,027 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,027 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,027 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,027 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,028 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,028 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,027 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,028 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,028 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:15,028 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,028 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. started... 2023-02-12 04:01:15,028 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 95c25e17def1a4a4a028fa2b6f6b5c57 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:01:15,039 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/0cb4668ee1584786a11c678e550f5f33 2023-02-12 04:01:15,047 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,048 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,048 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,048 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,048 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,048 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,051 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/f784af55cd14443aae9d0be9bf7b2901 2023-02-12 04:01:15,056 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/0cb4668ee1584786a11c678e550f5f33 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/0cb4668ee1584786a11c678e550f5f33 2023-02-12 04:01:15,060 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/0cb4668ee1584786a11c678e550f5f33, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:01:15,060 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/f784af55cd14443aae9d0be9bf7b2901 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/f784af55cd14443aae9d0be9bf7b2901 2023-02-12 04:01:15,087 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/f784af55cd14443aae9d0be9bf7b2901, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:01:15,088 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 95c25e17def1a4a4a028fa2b6f6b5c57 in 60ms, sequenceid=19, compaction requested=false 2023-02-12 04:01:15,088 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 95c25e17def1a4a4a028fa2b6f6b5c57: 2023-02-12 04:01:15,088 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:15,088 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:15,088 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:15,088 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,088 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,088 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,089 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,089 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,090 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,090 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,091 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-12 04:01:15,091 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6892573d[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:15,091 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-12 04:01:15,091 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,092 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,092 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,092 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,092 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,092 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,092 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,092 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,092 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,092 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:15,092 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:15,092 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:15,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,092 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,092 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,092 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:15,093 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,093 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,093 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,093 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,093 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,093 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,093 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,093 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,093 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,093 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:01:15,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:01:15,095 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-12 04:01:15,096 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:15,096 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:15,096 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,096 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:15,096 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,096 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,097 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,096 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,097 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,097 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,097 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,097 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,097 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,098 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,099 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,099 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,099 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,101 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,101 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,101 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,101 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,101 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,101 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,101 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,101 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,101 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,101 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,101 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,101 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:15,102 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:15,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,102 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,102 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,102 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,102 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,102 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,102 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,102 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,102 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:15,102 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:15,203 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:15,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:15,219 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:15,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:01:15,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,221 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:15,221 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,221 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-12 04:01:15,221 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-12 04:01:15,222 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,222 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,223 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,223 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,223 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,223 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,223 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,223 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:15,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,224 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,224 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,224 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,224 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,224 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,224 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,225 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,225 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,225 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,225 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,225 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:15,225 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:15,225 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,225 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,225 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:15,225 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,225 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. started... 2023-02-12 04:01:15,225 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 95c25e17def1a4a4a028fa2b6f6b5c57 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:01:15,227 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,227 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,227 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,227 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,227 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,227 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,236 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/632a83419e344fb585c8ec66b4e0df15 2023-02-12 04:01:15,251 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/55466f9431714326a0113c3aac89dbd7 2023-02-12 04:01:15,255 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/632a83419e344fb585c8ec66b4e0df15 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/632a83419e344fb585c8ec66b4e0df15 2023-02-12 04:01:15,260 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/632a83419e344fb585c8ec66b4e0df15, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:01:15,260 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/55466f9431714326a0113c3aac89dbd7 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/55466f9431714326a0113c3aac89dbd7 2023-02-12 04:01:15,264 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/55466f9431714326a0113c3aac89dbd7, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:01:15,265 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 95c25e17def1a4a4a028fa2b6f6b5c57 in 40ms, sequenceid=32, compaction requested=true 2023-02-12 04:01:15,265 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 95c25e17def1a4a4a028fa2b6f6b5c57: 2023-02-12 04:01:15,265 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:15,265 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:15,265 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:15,265 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,265 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,266 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,267 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,267 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,267 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,267 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-12 04:01:15,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5865d4bd[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:15,268 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-12 04:01:15,268 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,269 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,269 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,269 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,269 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,269 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,269 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,269 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:15,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,269 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,269 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,270 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,270 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:15,269 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:15,269 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,270 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:15,270 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,270 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,271 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,271 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,271 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,271 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,271 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,271 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,271 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,271 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:01:15,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:01:15,273 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-12 04:01:15,273 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:15,273 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:15,273 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,273 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:15,274 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,274 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,274 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,274 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,274 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,275 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,275 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,275 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,275 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,276 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,277 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,277 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,277 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,279 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,279 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,279 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,279 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,279 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,279 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,279 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,279 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,279 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,279 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,279 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,279 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,279 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:15,280 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:15,279 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,280 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,280 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,280 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:15,280 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,280 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,280 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:15,280 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,380 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:15,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:15,395 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:15,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:01:15,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,397 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:15,397 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,397 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-12 04:01:15,397 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-12 04:01:15,398 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,398 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,398 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,398 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,398 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,398 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,399 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,399 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:15,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,399 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,399 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,399 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,400 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,400 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,400 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,400 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,400 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,400 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,400 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,400 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,405 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,406 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,406 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:15,406 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,406 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:15,406 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,406 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,406 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:15,406 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. started... 2023-02-12 04:01:15,406 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,406 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,406 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,406 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,406 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 95c25e17def1a4a4a028fa2b6f6b5c57 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:01:15,406 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,406 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,406 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,406 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,406 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,414 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,414 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,414 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,414 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,415 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,415 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,415 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/43dbe4da36824373a7f19335758f79a5 2023-02-12 04:01:15,427 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/d16e42ddc0ec4b899039da0c47d40be3 2023-02-12 04:01:15,431 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/43dbe4da36824373a7f19335758f79a5 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/43dbe4da36824373a7f19335758f79a5 2023-02-12 04:01:15,435 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/43dbe4da36824373a7f19335758f79a5, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:01:15,436 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/d16e42ddc0ec4b899039da0c47d40be3 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d16e42ddc0ec4b899039da0c47d40be3 2023-02-12 04:01:15,440 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d16e42ddc0ec4b899039da0c47d40be3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:01:15,441 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 95c25e17def1a4a4a028fa2b6f6b5c57 in 35ms, sequenceid=45, compaction requested=true 2023-02-12 04:01:15,441 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 95c25e17def1a4a4a028fa2b6f6b5c57: 2023-02-12 04:01:15,441 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:15,441 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:15,441 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:15,441 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,441 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,442 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,443 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,443 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,443 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,443 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-12 04:01:15,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@755f44d9[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:15,444 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-12 04:01:15,444 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,445 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,445 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,445 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,445 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,445 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,445 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:15,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,445 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,445 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,445 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,445 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,445 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,445 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,445 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:15,446 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:15,447 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,447 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,447 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,448 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,448 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,448 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,449 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,449 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,449 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,449 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:01:15,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:01:15,451 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-12 04:01:15,451 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:15,451 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:15,451 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,451 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:15,452 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,452 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,453 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,453 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,453 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,453 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,453 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,453 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,453 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,454 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,455 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,455 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,456 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,460 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,460 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,460 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,460 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,461 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,461 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:15,461 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:15,460 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,461 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,461 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,461 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,461 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:15,461 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:15,461 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,461 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,461 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,462 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,462 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,462 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,462 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,462 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,462 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,561 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:15,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:15,574 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:15,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:01:15,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,576 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:15,576 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,576 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-12 04:01:15,576 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-12 04:01:15,577 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,577 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,577 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,577 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,577 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,577 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,578 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,578 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:15,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,578 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,578 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,578 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,578 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,579 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,579 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,579 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,579 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,579 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,579 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,579 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,579 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:15,579 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:15,579 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:15,580 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. started... 2023-02-12 04:01:15,580 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 95c25e17def1a4a4a028fa2b6f6b5c57 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:01:15,580 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,580 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,580 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,580 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,580 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,580 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,588 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/589166de011e4acaaaa158d64054b517 2023-02-12 04:01:15,599 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/d249d0621eda4ce68d78abab2f60dbd8 2023-02-12 04:01:15,604 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/589166de011e4acaaaa158d64054b517 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/589166de011e4acaaaa158d64054b517 2023-02-12 04:01:15,608 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/589166de011e4acaaaa158d64054b517, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:01:15,609 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/d249d0621eda4ce68d78abab2f60dbd8 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d249d0621eda4ce68d78abab2f60dbd8 2023-02-12 04:01:15,614 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d249d0621eda4ce68d78abab2f60dbd8, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:01:15,615 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 95c25e17def1a4a4a028fa2b6f6b5c57 in 35ms, sequenceid=58, compaction requested=true 2023-02-12 04:01:15,615 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 95c25e17def1a4a4a028fa2b6f6b5c57: 2023-02-12 04:01:15,615 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:15,615 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:15,615 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:15,615 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,615 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,615 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,616 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,616 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,617 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,617 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-12 04:01:15,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@769dd77b[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:15,618 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-12 04:01:15,618 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,619 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,619 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,619 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,619 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,619 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,619 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,620 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:15,619 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,620 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:15,619 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,620 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:15,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,620 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,620 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,620 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:15,621 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,621 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,621 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,621 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,621 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,621 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,621 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,621 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,621 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,624 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:01:15,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:01:15,627 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-12 04:01:15,627 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:15,627 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:15,627 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,627 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:15,628 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,628 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,628 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,628 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,628 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,628 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,628 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,628 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,628 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,629 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,630 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,630 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,630 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,633 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,633 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,633 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,633 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,633 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,633 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,633 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:15,634 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:15,633 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,633 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,633 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,634 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,633 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,634 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:15,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,634 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,634 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,634 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:15,634 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,634 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,634 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,634 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,634 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,734 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:15,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:15,747 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:15,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:01:15,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,748 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:15,748 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,748 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-12 04:01:15,748 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-12 04:01:15,749 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,749 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,749 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,750 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,749 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,750 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,750 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,750 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:15,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,750 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,750 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,750 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,751 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,751 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,751 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,751 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,751 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,751 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,751 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,751 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,751 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:15,751 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:15,752 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:15,752 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. started... 2023-02-12 04:01:15,752 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 95c25e17def1a4a4a028fa2b6f6b5c57 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:01:15,752 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,752 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,753 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,753 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,753 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,753 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,760 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/d21cc35ede794c61825466af65d78813 2023-02-12 04:01:15,772 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/7c6de596a85f4dc8baff667f5d0f856c 2023-02-12 04:01:15,776 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/d21cc35ede794c61825466af65d78813 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/d21cc35ede794c61825466af65d78813 2023-02-12 04:01:15,780 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/d21cc35ede794c61825466af65d78813, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:01:15,781 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/7c6de596a85f4dc8baff667f5d0f856c as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/7c6de596a85f4dc8baff667f5d0f856c 2023-02-12 04:01:15,785 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/7c6de596a85f4dc8baff667f5d0f856c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:01:15,786 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 95c25e17def1a4a4a028fa2b6f6b5c57 in 34ms, sequenceid=71, compaction requested=true 2023-02-12 04:01:15,786 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 95c25e17def1a4a4a028fa2b6f6b5c57: 2023-02-12 04:01:15,786 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:15,786 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:15,786 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:15,786 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,786 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,787 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,788 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,788 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,788 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,788 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-12 04:01:15,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c4c4d2a[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:15,790 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-12 04:01:15,790 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,790 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,790 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,790 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,790 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,790 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,791 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:15,790 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,791 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:15,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,791 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,791 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,791 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,791 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,791 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:15,791 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:15,792 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,792 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,792 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,792 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,792 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,792 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,792 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,792 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,792 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,792 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:01:15,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:01:15,794 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-12 04:01:15,794 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:15,794 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:15,794 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,794 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:15,795 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,795 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,795 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,796 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,796 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,796 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,796 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,796 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,796 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,797 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,798 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,798 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,798 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,800 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,800 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,800 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,800 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,800 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,800 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,800 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,800 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:15,800 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:15,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,800 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,800 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,800 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,800 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:15,800 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,800 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:15,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,800 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,801 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,801 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,801 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,801 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,801 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,801 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,901 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:15,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:15,915 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:15,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:01:15,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,917 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:15,917 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,917 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-12 04:01:15,917 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-12 04:01:15,918 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,918 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,919 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,919 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,919 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,919 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,919 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,919 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,919 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,919 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,919 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,920 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:15,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,920 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,920 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,920 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,920 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,920 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,920 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,920 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,920 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,920 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,920 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,920 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,920 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:15,920 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,920 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,920 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,920 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,920 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,920 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,920 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:15,920 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:15,920 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:15,921 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:15,921 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:15,921 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:15,921 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. started... 2023-02-12 04:01:15,921 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 95c25e17def1a4a4a028fa2b6f6b5c57 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-12 04:01:15,921 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,921 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,921 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,921 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,922 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,922 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,934 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/75ba2df51f65445bb3fe35f39be554a3 2023-02-12 04:01:15,937 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 75ba2df51f65445bb3fe35f39be554a3 2023-02-12 04:01:15,946 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/2e1b77030f3640ae8eda7f8812e6d1b1 2023-02-12 04:01:15,949 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e1b77030f3640ae8eda7f8812e6d1b1 2023-02-12 04:01:15,950 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/75ba2df51f65445bb3fe35f39be554a3 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/75ba2df51f65445bb3fe35f39be554a3 2023-02-12 04:01:15,954 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 75ba2df51f65445bb3fe35f39be554a3 2023-02-12 04:01:15,954 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/75ba2df51f65445bb3fe35f39be554a3, entries=9, sequenceid=83, filesize=5.1 K 2023-02-12 04:01:15,954 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/2e1b77030f3640ae8eda7f8812e6d1b1 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/2e1b77030f3640ae8eda7f8812e6d1b1 2023-02-12 04:01:15,959 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e1b77030f3640ae8eda7f8812e6d1b1 2023-02-12 04:01:15,959 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/2e1b77030f3640ae8eda7f8812e6d1b1, entries=9, sequenceid=83, filesize=5.1 K 2023-02-12 04:01:15,960 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 95c25e17def1a4a4a028fa2b6f6b5c57 in 38ms, sequenceid=83, compaction requested=true 2023-02-12 04:01:15,960 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 95c25e17def1a4a4a028fa2b6f6b5c57: 2023-02-12 04:01:15,960 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:15,960 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:15,960 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:15,960 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,960 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:15,960 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:15,961 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,961 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,961 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:15,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,963 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-12 04:01:15,963 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7afaa022[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:15,963 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-12 04:01:15,963 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,964 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,964 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,964 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,964 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,964 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:15,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,964 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,964 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,964 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:15,964 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,964 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,964 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:15,964 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:15,964 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:15,964 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:15,966 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,966 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,966 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,966 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,966 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,966 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,966 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,966 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:15,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,966 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,967 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:15,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:01:15,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:01:15,969 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-12 04:01:15,969 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:15,969 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:15,969 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,969 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:15,970 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,970 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:15,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:15,970 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,970 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,970 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:15,970 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,970 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,970 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,970 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:15,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,971 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:15,972 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,972 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,972 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,974 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:15,974 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:15,974 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:15,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:15,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:15,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:15,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:15,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:15,974 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,975 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:15,975 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:15,975 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,975 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:15,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:15,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:16,075 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:16,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:16,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:16,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:01:16,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,087 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:16,087 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:16,087 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-12 04:01:16,087 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-12 04:01:16,088 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,088 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,088 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:16,088 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:16,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:16,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:16,088 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:16,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:16,088 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:16,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:16,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:16,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,089 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,089 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:16,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:16,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,089 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:16,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,090 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:16,090 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:16,090 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:16,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:16,090 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:16,090 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:16,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:16,090 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:16,090 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:16,091 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. started... 2023-02-12 04:01:16,091 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 95c25e17def1a4a4a028fa2b6f6b5c57 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-12 04:01:16,091 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,091 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,091 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,091 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:16,091 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,092 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:16,098 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/2baf63f65b1a48f2a93ecc83b190612f 2023-02-12 04:01:16,102 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2baf63f65b1a48f2a93ecc83b190612f 2023-02-12 04:01:16,111 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/fd8e322672394efbbadd6bf33b9de27a 2023-02-12 04:01:16,114 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fd8e322672394efbbadd6bf33b9de27a 2023-02-12 04:01:16,115 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/2baf63f65b1a48f2a93ecc83b190612f as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/2baf63f65b1a48f2a93ecc83b190612f 2023-02-12 04:01:16,119 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2baf63f65b1a48f2a93ecc83b190612f 2023-02-12 04:01:16,119 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/2baf63f65b1a48f2a93ecc83b190612f, entries=5, sequenceid=91, filesize=5.0 K 2023-02-12 04:01:16,120 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/fd8e322672394efbbadd6bf33b9de27a as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/fd8e322672394efbbadd6bf33b9de27a 2023-02-12 04:01:16,124 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fd8e322672394efbbadd6bf33b9de27a 2023-02-12 04:01:16,124 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/fd8e322672394efbbadd6bf33b9de27a, entries=5, sequenceid=91, filesize=5.0 K 2023-02-12 04:01:16,125 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 95c25e17def1a4a4a028fa2b6f6b5c57 in 34ms, sequenceid=91, compaction requested=true 2023-02-12 04:01:16,125 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 95c25e17def1a4a4a028fa2b6f6b5c57: 2023-02-12 04:01:16,125 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:16,125 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:16,125 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:16,125 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,125 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:16,125 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:16,126 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,126 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:16,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:16,127 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,127 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:16,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:16,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:16,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:16,128 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-12 04:01:16,128 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@750c4a28[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:16,128 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-12 04:01:16,128 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,129 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,129 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,129 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,129 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,129 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:16,129 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:16,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,129 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:16,129 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:16,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,129 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:16,129 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:16,129 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:16,129 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:16,130 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:16,130 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:16,130 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,130 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:16,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:16,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:16,130 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:16,130 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:16,130 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:16,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:16,131 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:16,131 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:16,131 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:16,131 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:16,131 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:16,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:16,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:16,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:01:16,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:01:16,133 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-12 04:01:16,133 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:16,133 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:16,133 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,133 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:16,134 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,134 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,134 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,134 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:16,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:16,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:16,134 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,134 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:16,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,134 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:16,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:16,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:16,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:16,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:16,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:16,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:16,135 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:16,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,135 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:16,135 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:16,136 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,137 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,137 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,139 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,139 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,139 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:16,139 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:16,139 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,139 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:16,139 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:16,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:16,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:16,140 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:16,140 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:16,140 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:16,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:16,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:16,140 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:16,140 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,140 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,140 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:16,140 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,140 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:16,140 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:16,140 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:16,140 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:16,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:16,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:16,141 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:16,140 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:16,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:16,141 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:16,241 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:16,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:16,250 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:01:16,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:01:16,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,251 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:16,251 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:16,251 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-12 04:01:16,251 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-12 04:01:16,252 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,252 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,253 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:16,253 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:16,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:16,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:16,253 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,253 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:16,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:16,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:16,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:16,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:16,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,253 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,253 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:01:16,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,254 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:16,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,254 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:16,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,254 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:01:16,254 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:16,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,254 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,254 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:16,254 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:16,254 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,254 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:16,254 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,254 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:16,254 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:16,254 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:16,254 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-12 04:01:16,254 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:16,254 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:16,254 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:16,254 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:16,254 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,254 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:16,254 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-12 04:01:16,254 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:01:16,255 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-12 04:01:16,255 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:01:16,255 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:01:16,255 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:16,255 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. started... 2023-02-12 04:01:16,255 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 95c25e17def1a4a4a028fa2b6f6b5c57 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-12 04:01:16,255 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,255 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,256 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,256 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:16,256 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,256 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:16,263 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/721a9c427ec24e04a3272bcaac522792 2023-02-12 04:01:16,266 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 721a9c427ec24e04a3272bcaac522792 2023-02-12 04:01:16,274 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/6ffdcc6cbc8649789b91b335409b369b 2023-02-12 04:01:16,277 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ffdcc6cbc8649789b91b335409b369b 2023-02-12 04:01:16,278 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/721a9c427ec24e04a3272bcaac522792 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/721a9c427ec24e04a3272bcaac522792 2023-02-12 04:01:16,281 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 721a9c427ec24e04a3272bcaac522792 2023-02-12 04:01:16,281 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/721a9c427ec24e04a3272bcaac522792, entries=5, sequenceid=99, filesize=5.0 K 2023-02-12 04:01:16,282 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/6ffdcc6cbc8649789b91b335409b369b as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/6ffdcc6cbc8649789b91b335409b369b 2023-02-12 04:01:16,286 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ffdcc6cbc8649789b91b335409b369b 2023-02-12 04:01:16,286 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/6ffdcc6cbc8649789b91b335409b369b, entries=5, sequenceid=99, filesize=5.0 K 2023-02-12 04:01:16,287 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 95c25e17def1a4a4a028fa2b6f6b5c57 in 32ms, sequenceid=99, compaction requested=true 2023-02-12 04:01:16,287 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 95c25e17def1a4a4a028fa2b6f6b5c57: 2023-02-12 04:01:16,287 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:16,287 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:01:16,287 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:01:16,287 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,287 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-12 04:01:16,287 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-12 04:01:16,288 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,288 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:16,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:16,288 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,289 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:01:16,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:16,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:16,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:16,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-12 04:01:16,290 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-12 04:01:16,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6db26dfd[Count = 0] remaining members to acquire global barrier 2023-02-12 04:01:16,290 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,291 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,291 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,291 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,291 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:16,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,291 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,291 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:01:16,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,291 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:16,291 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-12 04:01:16,291 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:16,291 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:16,291 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:01:16,291 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-12 04:01:16,291 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:01:16,292 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:01:16,292 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:16,292 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:16,292 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:16,292 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:16,292 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:16,292 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,292 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-12 04:01:16,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:16,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:16,292 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:16,293 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:16,293 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-12 04:01:16,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:16,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:16,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:16,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:01:16,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:01:16,295 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-12 04:01:16,295 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:01:16,295 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:01:16,295 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,295 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:01:16,296 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,296 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:01:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:01:16,296 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,296 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,296 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:16,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,296 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,296 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:01:16,296 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:16,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:16,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:16,297 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:16,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:01:16,297 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:01:16,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,298 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,298 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,299 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,300 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:16,300 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:16,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:16,300 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:01:16,300 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-12 04:01:16,300 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:01:16,300 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:01:16,300 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:16,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:16,300 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:16,300 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:16,300 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:01:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:16,300 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:01:16,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:16,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:01:16,301 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:01:16,301 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,301 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,301 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,301 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:01:16,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:16,301 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:01:16,301 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:16,301 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:01:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:01:16,301 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:16,301 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,301 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:16,401 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:01:16,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:01:16,405 DEBUG [Listener at localhost.localdomain/35207] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,406 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:16,407 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:16,407 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:16,407 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:16,409 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=95c25e17def1a4a4a028fa2b6f6b5c57, UNASSIGN}] 2023-02-12 04:01:16,410 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=95c25e17def1a4a4a028fa2b6f6b5c57, UNASSIGN 2023-02-12 04:01:16,411 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=95c25e17def1a4a4a028fa2b6f6b5c57, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:16,411 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676174476411"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174476411"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174476411"}]},"ts":"1676174476411"} 2023-02-12 04:01:16,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 95c25e17def1a4a4a028fa2b6f6b5c57, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:16,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 95c25e17def1a4a4a028fa2b6f6b5c57, disabling compactions & flushes 2023-02-12 04:01:16,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:16,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:16,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. after waiting 0 ms 2023-02-12 04:01:16,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:16,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 95c25e17def1a4a4a028fa2b6f6b5c57 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-12 04:01:16,575 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/039432e5975f4300a37d41b04f59e09c 2023-02-12 04:01:16,579 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 039432e5975f4300a37d41b04f59e09c 2023-02-12 04:01:16,587 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/5c59ae7dc1fd45389a3d59c8669d67e7 2023-02-12 04:01:16,591 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c59ae7dc1fd45389a3d59c8669d67e7 2023-02-12 04:01:16,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf1/039432e5975f4300a37d41b04f59e09c as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/039432e5975f4300a37d41b04f59e09c 2023-02-12 04:01:16,595 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 039432e5975f4300a37d41b04f59e09c 2023-02-12 04:01:16,595 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/039432e5975f4300a37d41b04f59e09c, entries=1, sequenceid=103, filesize=4.9 K 2023-02-12 04:01:16,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.tmp/cf2/5c59ae7dc1fd45389a3d59c8669d67e7 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/5c59ae7dc1fd45389a3d59c8669d67e7 2023-02-12 04:01:16,599 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c59ae7dc1fd45389a3d59c8669d67e7 2023-02-12 04:01:16,599 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/5c59ae7dc1fd45389a3d59c8669d67e7, entries=1, sequenceid=103, filesize=4.9 K 2023-02-12 04:01:16,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 95c25e17def1a4a4a028fa2b6f6b5c57 in 35ms, sequenceid=103, compaction requested=true 2023-02-12 04:01:16,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-12 04:01:16,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:01:16,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. 2023-02-12 04:01:16,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 95c25e17def1a4a4a028fa2b6f6b5c57: Waiting for close lock at 1676174476565Running coprocessor pre-close hooks at 1676174476565Disabling compacts and flushes for region at 1676174476565Disabling writes for close at 1676174476565Obtaining lock to block concurrent updates at 1676174476565Preparing flush snapshotting stores in 95c25e17def1a4a4a028fa2b6f6b5c57 at 1676174476565Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1676174476565Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57. at 1676174476566 (+1 ms)Flushing 95c25e17def1a4a4a028fa2b6f6b5c57/cf1: creating writer at 1676174476567 (+1 ms)Flushing 95c25e17def1a4a4a028fa2b6f6b5c57/cf1: appending metadata at 1676174476569 (+2 ms)Flushing 95c25e17def1a4a4a028fa2b6f6b5c57/cf1: closing flushed file at 1676174476569Flushing 95c25e17def1a4a4a028fa2b6f6b5c57/cf2: creating writer at 1676174476579 (+10 ms)Flushing 95c25e17def1a4a4a028fa2b6f6b5c57/cf2: appending metadata at 1676174476581 (+2 ms)Flushing 95c25e17def1a4a4a028fa2b6f6b5c57/cf2: closing flushed file at 1676174476581Flushing 95c25e17def1a4a4a028fa2b6f6b5c57/cf1: reopening flushed file at 1676174476592 (+11 ms)Flushing 95c25e17def1a4a4a028fa2b6f6b5c57/cf2: reopening flushed file at 1676174476596 (+4 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 95c25e17def1a4a4a028fa2b6f6b5c57 in 35ms, sequenceid=103, compaction requested=true at 1676174476600 (+4 ms)Writing region close event to WAL at 1676174476602 (+2 ms)Running coprocessor post-close hooks at 1676174476605 (+3 ms)Closed at 1676174476605 2023-02-12 04:01:16,606 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,606 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=95c25e17def1a4a4a028fa2b6f6b5c57, regionState=CLOSED 2023-02-12 04:01:16,607 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676174476606"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174476606"}]},"ts":"1676174476606"} 2023-02-12 04:01:16,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-12 04:01:16,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 95c25e17def1a4a4a028fa2b6f6b5c57, server=jenkins-hbase20.apache.org,40151,1676174449139 in 196 msec 2023-02-12 04:01:16,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-12 04:01:16,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=95c25e17def1a4a4a028fa2b6f6b5c57, UNASSIGN in 200 msec 2023-02-12 04:01:16,621 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=95c25e17def1a4a4a028fa2b6f6b5c57, threads=16 2023-02-12 04:01:16,621 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/2e1b77030f3640ae8eda7f8812e6d1b1 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,621 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/55466f9431714326a0113c3aac89dbd7 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,623 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/5c59ae7dc1fd45389a3d59c8669d67e7 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,628 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/6ffdcc6cbc8649789b91b335409b369b for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,628 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/7c6de596a85f4dc8baff667f5d0f856c for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,631 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/cce1f75ff14d47488df98bf67c7ea161 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,631 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d16e42ddc0ec4b899039da0c47d40be3 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,631 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d249d0621eda4ce68d78abab2f60dbd8 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,635 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/f784af55cd14443aae9d0be9bf7b2901 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,639 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/fd8e322672394efbbadd6bf33b9de27a for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,640 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/039432e5975f4300a37d41b04f59e09c for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,642 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/7c6de596a85f4dc8baff667f5d0f856c, top=false 2023-02-12 04:01:16,645 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/0cb4668ee1584786a11c678e550f5f33 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,646 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/1b013602f2774d4cb46ddcd5f9afb5db for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,646 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/2baf63f65b1a48f2a93ecc83b190612f for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,650 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/43dbe4da36824373a7f19335758f79a5 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,650 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/589166de011e4acaaaa158d64054b517 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,652 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c59ae7dc1fd45389a3d59c8669d67e7 2023-02-12 04:01:16,653 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d249d0621eda4ce68d78abab2f60dbd8, top=false 2023-02-12 04:01:16,655 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d16e42ddc0ec4b899039da0c47d40be3, top=false 2023-02-12 04:01:16,655 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ffdcc6cbc8649789b91b335409b369b 2023-02-12 04:01:16,655 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e1b77030f3640ae8eda7f8812e6d1b1 2023-02-12 04:01:16,658 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 039432e5975f4300a37d41b04f59e09c 2023-02-12 04:01:16,659 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/589166de011e4acaaaa158d64054b517, top=false 2023-02-12 04:01:16,660 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fd8e322672394efbbadd6bf33b9de27a 2023-02-12 04:01:16,660 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/43dbe4da36824373a7f19335758f79a5, top=false 2023-02-12 04:01:16,663 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2baf63f65b1a48f2a93ecc83b190612f 2023-02-12 04:01:16,676 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c59ae7dc1fd45389a3d59c8669d67e7 2023-02-12 04:01:16,676 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/5c59ae7dc1fd45389a3d59c8669d67e7, top=true 2023-02-12 04:01:16,679 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2baf63f65b1a48f2a93ecc83b190612f 2023-02-12 04:01:16,679 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/2baf63f65b1a48f2a93ecc83b190612f, top=true 2023-02-12 04:01:16,679 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ffdcc6cbc8649789b91b335409b369b 2023-02-12 04:01:16,679 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/6ffdcc6cbc8649789b91b335409b369b, top=true 2023-02-12 04:01:16,680 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fd8e322672394efbbadd6bf33b9de27a 2023-02-12 04:01:16,680 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/fd8e322672394efbbadd6bf33b9de27a, top=true 2023-02-12 04:01:16,685 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.splits/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-7c6de596a85f4dc8baff667f5d0f856c for child: d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, parent: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,692 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 039432e5975f4300a37d41b04f59e09c 2023-02-12 04:01:16,692 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/039432e5975f4300a37d41b04f59e09c, top=true 2023-02-12 04:01:16,700 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/7c6de596a85f4dc8baff667f5d0f856c for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,700 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/632a83419e344fb585c8ec66b4e0df15 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,719 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.splits/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-589166de011e4acaaaa158d64054b517 for child: d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, parent: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,722 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/589166de011e4acaaaa158d64054b517 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,723 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/721a9c427ec24e04a3272bcaac522792 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,726 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 721a9c427ec24e04a3272bcaac522792 2023-02-12 04:01:16,729 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 721a9c427ec24e04a3272bcaac522792 2023-02-12 04:01:16,729 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/721a9c427ec24e04a3272bcaac522792, top=true 2023-02-12 04:01:16,735 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.splits/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d249d0621eda4ce68d78abab2f60dbd8 for child: d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, parent: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,736 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.splits/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-6ffdcc6cbc8649789b91b335409b369b for child: e342c5ec8229f3d46866d60ac95bc270, parent: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,736 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.splits/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-43dbe4da36824373a7f19335758f79a5 for child: d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, parent: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,736 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/6ffdcc6cbc8649789b91b335409b369b for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,736 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/75ba2df51f65445bb3fe35f39be554a3 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,740 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 75ba2df51f65445bb3fe35f39be554a3 2023-02-12 04:01:16,746 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e1b77030f3640ae8eda7f8812e6d1b1 2023-02-12 04:01:16,750 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/43dbe4da36824373a7f19335758f79a5 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,750 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/d21cc35ede794c61825466af65d78813 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,750 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.splits/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-039432e5975f4300a37d41b04f59e09c for child: e342c5ec8229f3d46866d60ac95bc270, parent: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,750 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/039432e5975f4300a37d41b04f59e09c for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,750 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.splits/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-2baf63f65b1a48f2a93ecc83b190612f for child: e342c5ec8229f3d46866d60ac95bc270, parent: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,750 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/2baf63f65b1a48f2a93ecc83b190612f for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,751 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d249d0621eda4ce68d78abab2f60dbd8 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,751 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.splits/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-fd8e322672394efbbadd6bf33b9de27a for child: e342c5ec8229f3d46866d60ac95bc270, parent: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,751 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/fd8e322672394efbbadd6bf33b9de27a for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,754 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/d21cc35ede794c61825466af65d78813, top=false 2023-02-12 04:01:16,763 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.splits/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-5c59ae7dc1fd45389a3d59c8669d67e7 for child: e342c5ec8229f3d46866d60ac95bc270, parent: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,763 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/5c59ae7dc1fd45389a3d59c8669d67e7 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,763 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.splits/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d16e42ddc0ec4b899039da0c47d40be3 for child: d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, parent: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,782 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d16e42ddc0ec4b899039da0c47d40be3 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,798 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.splits/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d21cc35ede794c61825466af65d78813 for child: d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, parent: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,806 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/.splits/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-721a9c427ec24e04a3272bcaac522792 for child: e342c5ec8229f3d46866d60ac95bc270, parent: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,806 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/721a9c427ec24e04a3272bcaac522792 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,812 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/d21cc35ede794c61825466af65d78813 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,816 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/cce1f75ff14d47488df98bf67c7ea161 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,819 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/f784af55cd14443aae9d0be9bf7b2901 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,831 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/55466f9431714326a0113c3aac89dbd7 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,843 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/2e1b77030f3640ae8eda7f8812e6d1b1 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,847 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/0cb4668ee1584786a11c678e550f5f33 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,847 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 75ba2df51f65445bb3fe35f39be554a3 2023-02-12 04:01:16,847 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/1b013602f2774d4cb46ddcd5f9afb5db for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,850 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/632a83419e344fb585c8ec66b4e0df15 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,853 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/75ba2df51f65445bb3fe35f39be554a3 for region: 95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:16,853 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 95c25e17def1a4a4a028fa2b6f6b5c57 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-12 04:01:16,890 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-12 04:01:16,893 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-12 04:01:16,894 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676174476894"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676174476894"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676174476894"}]},"ts":"1676174476894"} 2023-02-12 04:01:16,895 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676174476894"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174476894"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174476894"}]},"ts":"1676174476894"} 2023-02-12 04:01:16,895 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676174476894"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174476894"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174476894"}]},"ts":"1676174476894"} 2023-02-12 04:01:16,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e342c5ec8229f3d46866d60ac95bc270, ASSIGN}] 2023-02-12 04:01:16,901 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e342c5ec8229f3d46866d60ac95bc270, ASSIGN 2023-02-12 04:01:16,902 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e342c5ec8229f3d46866d60ac95bc270, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,40151,1676174449139; forceNewPlan=false, retain=false 2023-02-12 04:01:16,903 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, ASSIGN 2023-02-12 04:01:16,904 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,40151,1676174449139; forceNewPlan=false, retain=false 2023-02-12 04:01:17,052 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-12 04:01:17,053 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:17,054 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676174477053"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174477053"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174477053"}]},"ts":"1676174477053"} 2023-02-12 04:01:17,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:17,058 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=e342c5ec8229f3d46866d60ac95bc270, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:17,058 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676174477058"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174477058"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174477058"}]},"ts":"1676174477058"} 2023-02-12 04:01:17,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure e342c5ec8229f3d46866d60ac95bc270, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:17,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.', STARTKEY => '', ENDKEY => '60'} 2023-02-12 04:01:17,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:01:17,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 2023-02-12 04:01:17,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:17,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 2023-02-12 04:01:17,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 2023-02-12 04:01:17,211 INFO [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 2023-02-12 04:01:17,212 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1 2023-02-12 04:01:17,212 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1 2023-02-12 04:01:17,213 INFO [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 columnFamilyName cf1 2023-02-12 04:01:17,224 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/0cb4668ee1584786a11c678e550f5f33.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/0cb4668ee1584786a11c678e550f5f33-bottom 2023-02-12 04:01:17,228 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/1b013602f2774d4cb46ddcd5f9afb5db.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/1b013602f2774d4cb46ddcd5f9afb5db-bottom 2023-02-12 04:01:17,233 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/632a83419e344fb585c8ec66b4e0df15.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/632a83419e344fb585c8ec66b4e0df15-bottom 2023-02-12 04:01:17,237 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,238 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/75ba2df51f65445bb3fe35f39be554a3-bottom 2023-02-12 04:01:17,242 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-43dbe4da36824373a7f19335758f79a5 2023-02-12 04:01:17,245 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-589166de011e4acaaaa158d64054b517 2023-02-12 04:01:17,249 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d21cc35ede794c61825466af65d78813 2023-02-12 04:01:17,250 INFO [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(310): Store=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:17,250 INFO [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 2023-02-12 04:01:17,251 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2 2023-02-12 04:01:17,251 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2 2023-02-12 04:01:17,251 INFO [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 columnFamilyName cf2 2023-02-12 04:01:17,263 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,264 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/2e1b77030f3640ae8eda7f8812e6d1b1-bottom 2023-02-12 04:01:17,268 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/55466f9431714326a0113c3aac89dbd7.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/55466f9431714326a0113c3aac89dbd7-bottom 2023-02-12 04:01:17,272 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/cce1f75ff14d47488df98bf67c7ea161.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/cce1f75ff14d47488df98bf67c7ea161-bottom 2023-02-12 04:01:17,276 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/f784af55cd14443aae9d0be9bf7b2901.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/f784af55cd14443aae9d0be9bf7b2901-bottom 2023-02-12 04:01:17,278 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-7c6de596a85f4dc8baff667f5d0f856c 2023-02-12 04:01:17,281 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d16e42ddc0ec4b899039da0c47d40be3 2023-02-12 04:01:17,284 DEBUG [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d249d0621eda4ce68d78abab2f60dbd8 2023-02-12 04:01:17,285 INFO [StoreOpener-d5a95b4a9e2a7dc12ff3a61b5a4b7ab1-1] regionserver.HStore(310): Store=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:17,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 2023-02-12 04:01:17,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 2023-02-12 04:01:17,288 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:01:17,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 2023-02-12 04:01:17,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened d5a95b4a9e2a7dc12ff3a61b5a4b7ab1; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61728540, jitterRate=-0.0801730751991272}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:01:17,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for d5a95b4a9e2a7dc12ff3a61b5a4b7ab1: Running coprocessor pre-open hook at 1676174477210Writing region info on filesystem at 1676174477210Initializing all the Stores at 1676174477211 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174477211Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174477211Cleaning up temporary data from old regions at 1676174477287 (+76 ms)Cleaning up detritus from prior splits at 1676174477287Running coprocessor post-open hooks at 1676174477289 (+2 ms)Region opened successfully at 1676174477289 2023-02-12 04:01:17,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1., pid=77, masterSystemTime=1676174477207 2023-02-12 04:01:17,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:17,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:01:17,291 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:01:17,291 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:01:17,291 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,291 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,291 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1912): d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1 is initiating minor compaction (all files) 2023-02-12 04:01:17,291 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1912): d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2 is initiating minor compaction (all files) 2023-02-12 04:01:17,291 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,291 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegion(2258): Starting compaction of d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,292 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/1b013602f2774d4cb46ddcd5f9afb5db.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/1b013602f2774d4cb46ddcd5f9afb5db-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/0cb4668ee1584786a11c678e550f5f33.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/0cb4668ee1584786a11c678e550f5f33-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/632a83419e344fb585c8ec66b4e0df15.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/632a83419e344fb585c8ec66b4e0df15-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-43dbe4da36824373a7f19335758f79a5, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-589166de011e4acaaaa158d64054b517, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d21cc35ede794c61825466af65d78813, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/75ba2df51f65445bb3fe35f39be554a3-bottom] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/.tmp, totalSize=35.0 K 2023-02-12 04:01:17,292 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/cce1f75ff14d47488df98bf67c7ea161.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/cce1f75ff14d47488df98bf67c7ea161-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/f784af55cd14443aae9d0be9bf7b2901.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/f784af55cd14443aae9d0be9bf7b2901-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/55466f9431714326a0113c3aac89dbd7.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/55466f9431714326a0113c3aac89dbd7-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d16e42ddc0ec4b899039da0c47d40be3, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d249d0621eda4ce68d78abab2f60dbd8, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-7c6de596a85f4dc8baff667f5d0f856c, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/2e1b77030f3640ae8eda7f8812e6d1b1-bottom] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/.tmp, totalSize=35.0 K 2023-02-12 04:01:17,292 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting 1b013602f2774d4cb46ddcd5f9afb5db.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174474838 2023-02-12 04:01:17,292 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting cce1f75ff14d47488df98bf67c7ea161.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174474838 2023-02-12 04:01:17,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,292 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting 0cb4668ee1584786a11c678e550f5f33.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174475009 2023-02-12 04:01:17,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e342c5ec8229f3d46866d60ac95bc270, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.', STARTKEY => '60', ENDKEY => ''} 2023-02-12 04:01:17,293 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting f784af55cd14443aae9d0be9bf7b2901.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174475009 2023-02-12 04:01:17,293 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:17,293 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676174477293"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174477293"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174477293"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174477293"}]},"ts":"1676174477293"} 2023-02-12 04:01:17,293 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting 632a83419e344fb585c8ec66b4e0df15.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174475205 2023-02-12 04:01:17,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:01:17,293 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting 55466f9431714326a0113c3aac89dbd7.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174475205 2023-02-12 04:01:17,293 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-43dbe4da36824373a7f19335758f79a5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174475382 2023-02-12 04:01:17,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:17,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:17,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:17,294 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d16e42ddc0ec4b899039da0c47d40be3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174475382 2023-02-12 04:01:17,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:17,294 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-589166de011e4acaaaa158d64054b517, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174475563 2023-02-12 04:01:17,294 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d249d0621eda4ce68d78abab2f60dbd8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174475563 2023-02-12 04:01:17,294 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d21cc35ede794c61825466af65d78813, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174475736 2023-02-12 04:01:17,294 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-7c6de596a85f4dc8baff667f5d0f856c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174475736 2023-02-12 04:01:17,295 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting 75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676174475902 2023-02-12 04:01:17,295 INFO [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:17,295 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting 2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676174475902 2023-02-12 04:01:17,297 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1 2023-02-12 04:01:17,297 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1 2023-02-12 04:01:17,297 INFO [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e342c5ec8229f3d46866d60ac95bc270 columnFamilyName cf1 2023-02-12 04:01:17,297 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-12 04:01:17,297 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, server=jenkins-hbase20.apache.org,40151,1676174449139 in 239 msec 2023-02-12 04:01:17,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, ASSIGN in 398 msec 2023-02-12 04:01:17,311 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] throttle.PressureAwareThroughputController(145): d5a95b4a9e2a7dc12ff3a61b5a4b7ab1#cf1#compaction#72 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:17,312 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/0cb4668ee1584786a11c678e550f5f33.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/0cb4668ee1584786a11c678e550f5f33-top 2023-02-12 04:01:17,316 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/1b013602f2774d4cb46ddcd5f9afb5db.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/1b013602f2774d4cb46ddcd5f9afb5db-top 2023-02-12 04:01:17,319 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/632a83419e344fb585c8ec66b4e0df15.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/632a83419e344fb585c8ec66b4e0df15-top 2023-02-12 04:01:17,328 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,328 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/75ba2df51f65445bb3fe35f39be554a3-top 2023-02-12 04:01:17,328 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] throttle.PressureAwareThroughputController(145): d5a95b4a9e2a7dc12ff3a61b5a4b7ab1#cf2#compaction#73 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:17,332 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 039432e5975f4300a37d41b04f59e09c 2023-02-12 04:01:17,332 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-039432e5975f4300a37d41b04f59e09c 2023-02-12 04:01:17,336 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2baf63f65b1a48f2a93ecc83b190612f 2023-02-12 04:01:17,336 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-2baf63f65b1a48f2a93ecc83b190612f 2023-02-12 04:01:17,340 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/.tmp/cf1/060ec2a9c3a24a288e69396b69e6f73c as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/060ec2a9c3a24a288e69396b69e6f73c 2023-02-12 04:01:17,341 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 721a9c427ec24e04a3272bcaac522792 2023-02-12 04:01:17,341 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-721a9c427ec24e04a3272bcaac522792 2023-02-12 04:01:17,342 INFO [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(310): Store=e342c5ec8229f3d46866d60ac95bc270/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:17,342 INFO [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:17,342 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/.tmp/cf2/1adf5a3e2381445082d7a69d8675da09 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/1adf5a3e2381445082d7a69d8675da09 2023-02-12 04:01:17,342 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2 2023-02-12 04:01:17,343 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2 2023-02-12 04:01:17,343 INFO [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e342c5ec8229f3d46866d60ac95bc270 columnFamilyName cf2 2023-02-12 04:01:17,345 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1 of d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 into 060ec2a9c3a24a288e69396b69e6f73c(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:17,345 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for d5a95b4a9e2a7dc12ff3a61b5a4b7ab1: 2023-02-12 04:01:17,345 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1., storeName=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1, priority=9, startTime=1676174477290; duration=0sec 2023-02-12 04:01:17,345 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:17,347 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2 of d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 into 1adf5a3e2381445082d7a69d8675da09(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:17,347 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for d5a95b4a9e2a7dc12ff3a61b5a4b7ab1: 2023-02-12 04:01:17,347 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1., storeName=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2, priority=9, startTime=1676174477290; duration=0sec 2023-02-12 04:01:17,347 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:17,354 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,354 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/2e1b77030f3640ae8eda7f8812e6d1b1-top 2023-02-12 04:01:17,358 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/55466f9431714326a0113c3aac89dbd7.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/55466f9431714326a0113c3aac89dbd7-top 2023-02-12 04:01:17,362 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/cce1f75ff14d47488df98bf67c7ea161.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/cce1f75ff14d47488df98bf67c7ea161-top 2023-02-12 04:01:17,366 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/f784af55cd14443aae9d0be9bf7b2901.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/f784af55cd14443aae9d0be9bf7b2901-top 2023-02-12 04:01:17,369 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c59ae7dc1fd45389a3d59c8669d67e7 2023-02-12 04:01:17,369 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-5c59ae7dc1fd45389a3d59c8669d67e7 2023-02-12 04:01:17,372 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ffdcc6cbc8649789b91b335409b369b 2023-02-12 04:01:17,372 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-6ffdcc6cbc8649789b91b335409b369b 2023-02-12 04:01:17,375 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fd8e322672394efbbadd6bf33b9de27a 2023-02-12 04:01:17,376 DEBUG [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-fd8e322672394efbbadd6bf33b9de27a 2023-02-12 04:01:17,376 INFO [StoreOpener-e342c5ec8229f3d46866d60ac95bc270-1] regionserver.HStore(310): Store=e342c5ec8229f3d46866d60ac95bc270/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:17,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:17,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:17,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:01:17,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:17,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened e342c5ec8229f3d46866d60ac95bc270; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64847822, jitterRate=-0.03369215130805969}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:01:17,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for e342c5ec8229f3d46866d60ac95bc270: Running coprocessor pre-open hook at 1676174477294Writing region info on filesystem at 1676174477294Initializing all the Stores at 1676174477294Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174477294Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174477295 (+1 ms)Cleaning up temporary data from old regions at 1676174477378 (+83 ms)Cleaning up detritus from prior splits at 1676174477378Running coprocessor post-open hooks at 1676174477381 (+3 ms)Region opened successfully at 1676174477381 2023-02-12 04:01:17,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270., pid=78, masterSystemTime=1676174477207 2023-02-12 04:01:17,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:17,383 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:01:17,383 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:01:17,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:01:17,383 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,383 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,383 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1912): e342c5ec8229f3d46866d60ac95bc270/cf2 is initiating minor compaction (all files) 2023-02-12 04:01:17,383 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1912): e342c5ec8229f3d46866d60ac95bc270/cf1 is initiating minor compaction (all files) 2023-02-12 04:01:17,383 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e342c5ec8229f3d46866d60ac95bc270/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,383 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e342c5ec8229f3d46866d60ac95bc270/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,383 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/1b013602f2774d4cb46ddcd5f9afb5db.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/1b013602f2774d4cb46ddcd5f9afb5db-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/0cb4668ee1584786a11c678e550f5f33.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/0cb4668ee1584786a11c678e550f5f33-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/632a83419e344fb585c8ec66b4e0df15.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/632a83419e344fb585c8ec66b4e0df15-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/75ba2df51f65445bb3fe35f39be554a3-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-2baf63f65b1a48f2a93ecc83b190612f, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-721a9c427ec24e04a3272bcaac522792, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-039432e5975f4300a37d41b04f59e09c] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/.tmp, totalSize=34.8 K 2023-02-12 04:01:17,383 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/cce1f75ff14d47488df98bf67c7ea161.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/cce1f75ff14d47488df98bf67c7ea161-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/f784af55cd14443aae9d0be9bf7b2901.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/f784af55cd14443aae9d0be9bf7b2901-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/55466f9431714326a0113c3aac89dbd7.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/55466f9431714326a0113c3aac89dbd7-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/2e1b77030f3640ae8eda7f8812e6d1b1-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-fd8e322672394efbbadd6bf33b9de27a, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-6ffdcc6cbc8649789b91b335409b369b, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-5c59ae7dc1fd45389a3d59c8669d67e7] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/.tmp, totalSize=34.8 K 2023-02-12 04:01:17,384 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting 1b013602f2774d4cb46ddcd5f9afb5db.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676174474838 2023-02-12 04:01:17,384 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting cce1f75ff14d47488df98bf67c7ea161.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676174474838 2023-02-12 04:01:17,384 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting 0cb4668ee1584786a11c678e550f5f33.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676174475009 2023-02-12 04:01:17,384 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting f784af55cd14443aae9d0be9bf7b2901.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676174475009 2023-02-12 04:01:17,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,385 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=e342c5ec8229f3d46866d60ac95bc270, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:17,385 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting 632a83419e344fb585c8ec66b4e0df15.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676174475205 2023-02-12 04:01:17,385 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting 55466f9431714326a0113c3aac89dbd7.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676174475205 2023-02-12 04:01:17,385 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676174477385"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174477385"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174477385"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174477385"}]},"ts":"1676174477385"} 2023-02-12 04:01:17,385 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting 75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676174475902 2023-02-12 04:01:17,385 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting 2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676174475902 2023-02-12 04:01:17,386 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-2baf63f65b1a48f2a93ecc83b190612f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-12 04:01:17,386 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-fd8e322672394efbbadd6bf33b9de27a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-12 04:01:17,386 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-721a9c427ec24e04a3272bcaac522792, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-12 04:01:17,386 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-6ffdcc6cbc8649789b91b335409b369b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-12 04:01:17,386 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-039432e5975f4300a37d41b04f59e09c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-12 04:01:17,386 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-5c59ae7dc1fd45389a3d59c8669d67e7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-12 04:01:17,388 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-12 04:01:17,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure e342c5ec8229f3d46866d60ac95bc270, server=jenkins-hbase20.apache.org,40151,1676174449139 in 327 msec 2023-02-12 04:01:17,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-12 04:01:17,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e342c5ec8229f3d46866d60ac95bc270, ASSIGN in 489 msec 2023-02-12 04:01:17,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 in 985 msec 2023-02-12 04:01:17,400 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] throttle.PressureAwareThroughputController(145): e342c5ec8229f3d46866d60ac95bc270#cf1#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:17,400 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] throttle.PressureAwareThroughputController(145): e342c5ec8229f3d46866d60ac95bc270#cf2#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:17,426 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 8b357229771e4ed4bc4390875e4bee7d: NONE, but ROW specified in column family configuration 2023-02-12 04:01:17,427 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/.tmp/cf2/8b357229771e4ed4bc4390875e4bee7d as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/8b357229771e4ed4bc4390875e4bee7d 2023-02-12 04:01:17,427 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-12 04:01:17,429 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 5259b42d1cd745f0b09aa7ccb8d3d44f: NONE, but ROW specified in column family configuration 2023-02-12 04:01:17,430 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/.tmp/cf1/5259b42d1cd745f0b09aa7ccb8d3d44f as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/5259b42d1cd745f0b09aa7ccb8d3d44f 2023-02-12 04:01:17,432 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 8b357229771e4ed4bc4390875e4bee7d: NONE, but ROW specified in column family configuration 2023-02-12 04:01:17,433 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e342c5ec8229f3d46866d60ac95bc270/cf2 of e342c5ec8229f3d46866d60ac95bc270 into 8b357229771e4ed4bc4390875e4bee7d(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:17,433 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e342c5ec8229f3d46866d60ac95bc270: 2023-02-12 04:01:17,433 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270., storeName=e342c5ec8229f3d46866d60ac95bc270/cf2, priority=9, startTime=1676174477383; duration=0sec 2023-02-12 04:01:17,433 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:17,434 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 5259b42d1cd745f0b09aa7ccb8d3d44f: NONE, but ROW specified in column family configuration 2023-02-12 04:01:17,435 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e342c5ec8229f3d46866d60ac95bc270/cf1 of e342c5ec8229f3d46866d60ac95bc270 into 5259b42d1cd745f0b09aa7ccb8d3d44f(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:17,435 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e342c5ec8229f3d46866d60ac95bc270: 2023-02-12 04:01:17,435 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270., storeName=e342c5ec8229f3d46866d60ac95bc270/cf1, priority=9, startTime=1676174477382; duration=0sec 2023-02-12 04:01:17,435 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:17,443 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:53732, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-12 04:01:17,444 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40151] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,445 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40151] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-12 04:01:17,445 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40151] regionserver.HStore(1912): d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1 is initiating major compaction (all files) 2023-02-12 04:01:17,445 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40151] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@42eca1fd; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:17,445 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegion(2258): Starting compaction of d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,445 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40151] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-12 04:01:17,445 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/060ec2a9c3a24a288e69396b69e6f73c] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/.tmp, totalSize=6.8 K 2023-02-12 04:01:17,445 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40151] regionserver.HStore(1912): d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2 is initiating major compaction (all files) 2023-02-12 04:01:17,445 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40151] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@eef648d; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:17,445 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,445 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/1adf5a3e2381445082d7a69d8675da09] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/.tmp, totalSize=6.8 K 2023-02-12 04:01:17,446 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting 060ec2a9c3a24a288e69396b69e6f73c, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676174474838 2023-02-12 04:01:17,446 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting 1adf5a3e2381445082d7a69d8675da09, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676174474838 2023-02-12 04:01:17,447 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40151] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,447 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40151] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-12 04:01:17,448 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40151] regionserver.HStore(1912): e342c5ec8229f3d46866d60ac95bc270/cf1 is initiating major compaction (all files) 2023-02-12 04:01:17,448 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40151] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@38b47c9d; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:01:17,448 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40151] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-12 04:01:17,448 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40151] regionserver.HStore(1912): e342c5ec8229f3d46866d60ac95bc270/cf2 is initiating major compaction (all files) 2023-02-12 04:01:17,448 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40151] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@166f7e6a; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-12 04:01:17,449 INFO [Listener at localhost.localdomain/35207] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-12 04:01:17,450 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] throttle.PressureAwareThroughputController(145): d5a95b4a9e2a7dc12ff3a61b5a4b7ab1#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:17,450 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] throttle.PressureAwareThroughputController(145): d5a95b4a9e2a7dc12ff3a61b5a4b7ab1#cf2#compaction#77 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:17,468 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40151] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. is not mergeable because it has references 2023-02-12 04:01:17,468 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/.tmp/cf1/e9890106e6c24f2aacfa40ea598ddf3c as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/e9890106e6c24f2aacfa40ea598ddf3c 2023-02-12 04:01:17,469 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40151] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. is not mergeable because it has references 2023-02-12 04:01:17,474 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/.tmp/cf2/f6074148eb7e4f9299edd54eb57964d3 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/f6074148eb7e4f9299edd54eb57964d3 2023-02-12 04:01:17,474 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1 of d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 into e9890106e6c24f2aacfa40ea598ddf3c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:17,474 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for d5a95b4a9e2a7dc12ff3a61b5a4b7ab1: 2023-02-12 04:01:17,475 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676174477445; duration=0sec 2023-02-12 04:01:17,476 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-12 04:01:17,476 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e342c5ec8229f3d46866d60ac95bc270/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,476 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/5259b42d1cd745f0b09aa7ccb8d3d44f] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/.tmp, totalSize=5.0 K 2023-02-12 04:01:17,476 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting 5259b42d1cd745f0b09aa7ccb8d3d44f, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-12 04:01:17,478 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2 of d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 into f6074148eb7e4f9299edd54eb57964d3(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:17,479 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for d5a95b4a9e2a7dc12ff3a61b5a4b7ab1: 2023-02-12 04:01:17,479 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] throttle.PressureAwareThroughputController(145): e342c5ec8229f3d46866d60ac95bc270#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:17,479 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676174477445; duration=0sec 2023-02-12 04:01:17,479 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:01:17,479 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e342c5ec8229f3d46866d60ac95bc270/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,479 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/8b357229771e4ed4bc4390875e4bee7d] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/.tmp, totalSize=5.0 K 2023-02-12 04:01:17,479 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting 8b357229771e4ed4bc4390875e4bee7d, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-12 04:01:17,482 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] throttle.PressureAwareThroughputController(145): e342c5ec8229f3d46866d60ac95bc270#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:01:17,493 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 15085828a4954acea8bce28fd7d42822: NONE, but ROW specified in column family configuration 2023-02-12 04:01:17,493 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/.tmp/cf1/15085828a4954acea8bce28fd7d42822 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/15085828a4954acea8bce28fd7d42822 2023-02-12 04:01:17,495 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for dc4847bd0c624a65a3a3fb2d4cd8f65a: NONE, but ROW specified in column family configuration 2023-02-12 04:01:17,495 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/.tmp/cf2/dc4847bd0c624a65a3a3fb2d4cd8f65a as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/dc4847bd0c624a65a3a3fb2d4cd8f65a 2023-02-12 04:01:17,497 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 15085828a4954acea8bce28fd7d42822: NONE, but ROW specified in column family configuration 2023-02-12 04:01:17,498 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in e342c5ec8229f3d46866d60ac95bc270/cf1 of e342c5ec8229f3d46866d60ac95bc270 into 15085828a4954acea8bce28fd7d42822(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:17,498 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e342c5ec8229f3d46866d60ac95bc270: 2023-02-12 04:01:17,499 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676174477447; duration=0sec 2023-02-12 04:01:17,499 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:17,500 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for dc4847bd0c624a65a3a3fb2d4cd8f65a: NONE, but ROW specified in column family configuration 2023-02-12 04:01:17,500 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in e342c5ec8229f3d46866d60ac95bc270/cf2 of e342c5ec8229f3d46866d60ac95bc270 into dc4847bd0c624a65a3a3fb2d4cd8f65a(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:01:17,501 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e342c5ec8229f3d46866d60ac95bc270: 2023-02-12 04:01:17,501 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676174477448; duration=0sec 2023-02-12 04:01:17,501 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:01:17,512 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 04:01:17,578 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40151] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. is not mergeable because it has references 2023-02-12 04:01:17,579 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40151] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. is not mergeable because it has references 2023-02-12 04:01:17,586 DEBUG [Listener at localhost.localdomain/35207] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-12 04:01:17,587 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:01:17,589 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:17,589 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:17,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:17,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-12 04:01:17,593 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174477593"}]},"ts":"1676174477593"} 2023-02-12 04:01:17,594 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-12 04:01:17,595 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-12 04:01:17,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e342c5ec8229f3d46866d60ac95bc270, UNASSIGN}] 2023-02-12 04:01:17,597 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e342c5ec8229f3d46866d60ac95bc270, UNASSIGN 2023-02-12 04:01:17,597 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=e342c5ec8229f3d46866d60ac95bc270, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:17,597 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676174477597"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174477597"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174477597"}]},"ts":"1676174477597"} 2023-02-12 04:01:17,599 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure e342c5ec8229f3d46866d60ac95bc270, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:17,601 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, UNASSIGN 2023-02-12 04:01:17,601 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:17,601 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676174477601"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174477601"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174477601"}]},"ts":"1676174477601"} 2023-02-12 04:01:17,603 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:17,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-12 04:01:17,751 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:17,751 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing e342c5ec8229f3d46866d60ac95bc270, disabling compactions & flushes 2023-02-12 04:01:17,751 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,751 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,751 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. after waiting 0 ms 2023-02-12 04:01:17,751 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,751 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/1b013602f2774d4cb46ddcd5f9afb5db.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/1b013602f2774d4cb46ddcd5f9afb5db-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/0cb4668ee1584786a11c678e550f5f33.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/0cb4668ee1584786a11c678e550f5f33-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/632a83419e344fb585c8ec66b4e0df15.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/632a83419e344fb585c8ec66b4e0df15-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/75ba2df51f65445bb3fe35f39be554a3-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-2baf63f65b1a48f2a93ecc83b190612f, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-721a9c427ec24e04a3272bcaac522792, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/5259b42d1cd745f0b09aa7ccb8d3d44f, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-039432e5975f4300a37d41b04f59e09c] to archive 2023-02-12 04:01:17,752 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:01:17,754 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/1b013602f2774d4cb46ddcd5f9afb5db.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/1b013602f2774d4cb46ddcd5f9afb5db.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,755 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/0cb4668ee1584786a11c678e550f5f33.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/0cb4668ee1584786a11c678e550f5f33.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,756 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/632a83419e344fb585c8ec66b4e0df15.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/632a83419e344fb585c8ec66b4e0df15.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,757 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,759 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-2baf63f65b1a48f2a93ecc83b190612f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-2baf63f65b1a48f2a93ecc83b190612f 2023-02-12 04:01:17,760 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-721a9c427ec24e04a3272bcaac522792 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-721a9c427ec24e04a3272bcaac522792 2023-02-12 04:01:17,761 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/5259b42d1cd745f0b09aa7ccb8d3d44f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/5259b42d1cd745f0b09aa7ccb8d3d44f 2023-02-12 04:01:17,762 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-039432e5975f4300a37d41b04f59e09c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-039432e5975f4300a37d41b04f59e09c 2023-02-12 04:01:17,763 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/cce1f75ff14d47488df98bf67c7ea161.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/cce1f75ff14d47488df98bf67c7ea161-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/f784af55cd14443aae9d0be9bf7b2901.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/f784af55cd14443aae9d0be9bf7b2901-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/55466f9431714326a0113c3aac89dbd7.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/55466f9431714326a0113c3aac89dbd7-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/2e1b77030f3640ae8eda7f8812e6d1b1-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-fd8e322672394efbbadd6bf33b9de27a, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-6ffdcc6cbc8649789b91b335409b369b, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/8b357229771e4ed4bc4390875e4bee7d, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-5c59ae7dc1fd45389a3d59c8669d67e7] to archive 2023-02-12 04:01:17,763 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:01:17,765 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/cce1f75ff14d47488df98bf67c7ea161.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/cce1f75ff14d47488df98bf67c7ea161.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,765 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/f784af55cd14443aae9d0be9bf7b2901.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/f784af55cd14443aae9d0be9bf7b2901.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,766 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/55466f9431714326a0113c3aac89dbd7.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/55466f9431714326a0113c3aac89dbd7.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,767 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,768 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-fd8e322672394efbbadd6bf33b9de27a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-fd8e322672394efbbadd6bf33b9de27a 2023-02-12 04:01:17,769 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-6ffdcc6cbc8649789b91b335409b369b to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-6ffdcc6cbc8649789b91b335409b369b 2023-02-12 04:01:17,770 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/8b357229771e4ed4bc4390875e4bee7d to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/8b357229771e4ed4bc4390875e4bee7d 2023-02-12 04:01:17,771 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-5c59ae7dc1fd45389a3d59c8669d67e7 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-5c59ae7dc1fd45389a3d59c8669d67e7 2023-02-12 04:01:17,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-12 04:01:17,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:01:17,776 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270. 2023-02-12 04:01:17,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for e342c5ec8229f3d46866d60ac95bc270: Waiting for close lock at 1676174477751Running coprocessor pre-close hooks at 1676174477751Disabling compacts and flushes for region at 1676174477751Disabling writes for close at 1676174477751Writing region close event to WAL at 1676174477772 (+21 ms)Running coprocessor post-close hooks at 1676174477775 (+3 ms)Closed at 1676174477776 (+1 ms) 2023-02-12 04:01:17,777 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:17,777 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 2023-02-12 04:01:17,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, disabling compactions & flushes 2023-02-12 04:01:17,777 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. after waiting 0 ms 2023-02-12 04:01:17,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,778 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=e342c5ec8229f3d46866d60ac95bc270, regionState=CLOSED 2023-02-12 04:01:17,778 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676174477778"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174477778"}]},"ts":"1676174477778"} 2023-02-12 04:01:17,778 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/1b013602f2774d4cb46ddcd5f9afb5db.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/1b013602f2774d4cb46ddcd5f9afb5db-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/0cb4668ee1584786a11c678e550f5f33.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/0cb4668ee1584786a11c678e550f5f33-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/632a83419e344fb585c8ec66b4e0df15.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/632a83419e344fb585c8ec66b4e0df15-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-43dbe4da36824373a7f19335758f79a5, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-589166de011e4acaaaa158d64054b517, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d21cc35ede794c61825466af65d78813, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/060ec2a9c3a24a288e69396b69e6f73c, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/75ba2df51f65445bb3fe35f39be554a3-bottom] to archive 2023-02-12 04:01:17,779 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:01:17,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-12 04:01:17,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure e342c5ec8229f3d46866d60ac95bc270, server=jenkins-hbase20.apache.org,40151,1676174449139 in 180 msec 2023-02-12 04:01:17,781 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/1b013602f2774d4cb46ddcd5f9afb5db.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/1b013602f2774d4cb46ddcd5f9afb5db.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e342c5ec8229f3d46866d60ac95bc270, UNASSIGN in 185 msec 2023-02-12 04:01:17,782 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/0cb4668ee1584786a11c678e550f5f33.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/0cb4668ee1584786a11c678e550f5f33.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,783 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/632a83419e344fb585c8ec66b4e0df15.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/632a83419e344fb585c8ec66b4e0df15.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,783 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-43dbe4da36824373a7f19335758f79a5 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-43dbe4da36824373a7f19335758f79a5 2023-02-12 04:01:17,784 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-589166de011e4acaaaa158d64054b517 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-589166de011e4acaaaa158d64054b517 2023-02-12 04:01:17,785 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d21cc35ede794c61825466af65d78813 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d21cc35ede794c61825466af65d78813 2023-02-12 04:01:17,786 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/060ec2a9c3a24a288e69396b69e6f73c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/060ec2a9c3a24a288e69396b69e6f73c 2023-02-12 04:01:17,787 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/75ba2df51f65445bb3fe35f39be554a3.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,789 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/cce1f75ff14d47488df98bf67c7ea161.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/cce1f75ff14d47488df98bf67c7ea161-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/f784af55cd14443aae9d0be9bf7b2901.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/f784af55cd14443aae9d0be9bf7b2901-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/55466f9431714326a0113c3aac89dbd7.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/55466f9431714326a0113c3aac89dbd7-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d16e42ddc0ec4b899039da0c47d40be3, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d249d0621eda4ce68d78abab2f60dbd8, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-7c6de596a85f4dc8baff667f5d0f856c, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/1adf5a3e2381445082d7a69d8675da09, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/2e1b77030f3640ae8eda7f8812e6d1b1-bottom] to archive 2023-02-12 04:01:17,789 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:01:17,790 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/cce1f75ff14d47488df98bf67c7ea161.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/cce1f75ff14d47488df98bf67c7ea161.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,791 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/f784af55cd14443aae9d0be9bf7b2901.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/f784af55cd14443aae9d0be9bf7b2901.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,792 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/55466f9431714326a0113c3aac89dbd7.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/55466f9431714326a0113c3aac89dbd7.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,793 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d16e42ddc0ec4b899039da0c47d40be3 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d16e42ddc0ec4b899039da0c47d40be3 2023-02-12 04:01:17,794 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d249d0621eda4ce68d78abab2f60dbd8 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-d249d0621eda4ce68d78abab2f60dbd8 2023-02-12 04:01:17,795 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-7c6de596a85f4dc8baff667f5d0f856c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/testSplitTableRegionDeletedRowsDaughter=95c25e17def1a4a4a028fa2b6f6b5c57-7c6de596a85f4dc8baff667f5d0f856c 2023-02-12 04:01:17,796 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/1adf5a3e2381445082d7a69d8675da09 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/1adf5a3e2381445082d7a69d8675da09 2023-02-12 04:01:17,796 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/2e1b77030f3640ae8eda7f8812e6d1b1.95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,800 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-12 04:01:17,800 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:01:17,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1. 2023-02-12 04:01:17,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for d5a95b4a9e2a7dc12ff3a61b5a4b7ab1: Waiting for close lock at 1676174477777Running coprocessor pre-close hooks at 1676174477777Disabling compacts and flushes for region at 1676174477777Disabling writes for close at 1676174477778 (+1 ms)Writing region close event to WAL at 1676174477797 (+19 ms)Running coprocessor post-close hooks at 1676174477800 (+3 ms)Closed at 1676174477801 (+1 ms) 2023-02-12 04:01:17,802 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 2023-02-12 04:01:17,802 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, regionState=CLOSED 2023-02-12 04:01:17,803 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676174477802"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174477802"}]},"ts":"1676174477802"} 2023-02-12 04:01:17,805 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-12 04:01:17,805 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, server=jenkins-hbase20.apache.org,40151,1676174449139 in 202 msec 2023-02-12 04:01:17,806 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-12 04:01:17,806 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, UNASSIGN in 209 msec 2023-02-12 04:01:17,807 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174477807"}]},"ts":"1676174477807"} 2023-02-12 04:01:17,808 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-12 04:01:17,809 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-12 04:01:17,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 221 msec 2023-02-12 04:01:17,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-12 04:01:17,895 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-12 04:01:17,896 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:17,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:17,898 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:17,899 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:17,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-12 04:01:17,903 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,903 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 2023-02-12 04:01:17,903 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:17,905 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/recovered.edits] 2023-02-12 04:01:17,905 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/recovered.edits] 2023-02-12 04:01:17,905 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/recovered.edits] 2023-02-12 04:01:17,908 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/e9890106e6c24f2aacfa40ea598ddf3c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf1/e9890106e6c24f2aacfa40ea598ddf3c 2023-02-12 04:01:17,908 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/15085828a4954acea8bce28fd7d42822 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf1/15085828a4954acea8bce28fd7d42822 2023-02-12 04:01:17,908 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/039432e5975f4300a37d41b04f59e09c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/039432e5975f4300a37d41b04f59e09c 2023-02-12 04:01:17,909 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/0cb4668ee1584786a11c678e550f5f33 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/0cb4668ee1584786a11c678e550f5f33 2023-02-12 04:01:17,909 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/f6074148eb7e4f9299edd54eb57964d3 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/cf2/f6074148eb7e4f9299edd54eb57964d3 2023-02-12 04:01:17,910 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/dc4847bd0c624a65a3a3fb2d4cd8f65a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/cf2/dc4847bd0c624a65a3a3fb2d4cd8f65a 2023-02-12 04:01:17,910 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/1b013602f2774d4cb46ddcd5f9afb5db to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/1b013602f2774d4cb46ddcd5f9afb5db 2023-02-12 04:01:17,911 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/2baf63f65b1a48f2a93ecc83b190612f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/2baf63f65b1a48f2a93ecc83b190612f 2023-02-12 04:01:17,912 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/recovered.edits/113.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1/recovered.edits/113.seqid 2023-02-12 04:01:17,912 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/43dbe4da36824373a7f19335758f79a5 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/43dbe4da36824373a7f19335758f79a5 2023-02-12 04:01:17,912 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d5a95b4a9e2a7dc12ff3a61b5a4b7ab1 2023-02-12 04:01:17,912 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/recovered.edits/113.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270/recovered.edits/113.seqid 2023-02-12 04:01:17,913 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:01:17,913 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/589166de011e4acaaaa158d64054b517 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/589166de011e4acaaaa158d64054b517 2023-02-12 04:01:17,914 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/632a83419e344fb585c8ec66b4e0df15 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/632a83419e344fb585c8ec66b4e0df15 2023-02-12 04:01:17,915 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/721a9c427ec24e04a3272bcaac522792 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/721a9c427ec24e04a3272bcaac522792 2023-02-12 04:01:17,916 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/75ba2df51f65445bb3fe35f39be554a3 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/75ba2df51f65445bb3fe35f39be554a3 2023-02-12 04:01:17,917 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/d21cc35ede794c61825466af65d78813 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf1/d21cc35ede794c61825466af65d78813 2023-02-12 04:01:17,919 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/2e1b77030f3640ae8eda7f8812e6d1b1 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/2e1b77030f3640ae8eda7f8812e6d1b1 2023-02-12 04:01:17,920 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/55466f9431714326a0113c3aac89dbd7 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/55466f9431714326a0113c3aac89dbd7 2023-02-12 04:01:17,921 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/5c59ae7dc1fd45389a3d59c8669d67e7 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/5c59ae7dc1fd45389a3d59c8669d67e7 2023-02-12 04:01:17,922 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/6ffdcc6cbc8649789b91b335409b369b to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/6ffdcc6cbc8649789b91b335409b369b 2023-02-12 04:01:17,923 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/7c6de596a85f4dc8baff667f5d0f856c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/7c6de596a85f4dc8baff667f5d0f856c 2023-02-12 04:01:17,924 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/cce1f75ff14d47488df98bf67c7ea161 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/cce1f75ff14d47488df98bf67c7ea161 2023-02-12 04:01:17,924 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d16e42ddc0ec4b899039da0c47d40be3 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d16e42ddc0ec4b899039da0c47d40be3 2023-02-12 04:01:17,925 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d249d0621eda4ce68d78abab2f60dbd8 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/d249d0621eda4ce68d78abab2f60dbd8 2023-02-12 04:01:17,926 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/f784af55cd14443aae9d0be9bf7b2901 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/f784af55cd14443aae9d0be9bf7b2901 2023-02-12 04:01:17,927 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/fd8e322672394efbbadd6bf33b9de27a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/cf2/fd8e322672394efbbadd6bf33b9de27a 2023-02-12 04:01:17,930 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/recovered.edits/106.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57/recovered.edits/106.seqid 2023-02-12 04:01:17,930 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/95c25e17def1a4a4a028fa2b6f6b5c57 2023-02-12 04:01:17,930 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-12 04:01:17,932 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:17,934 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-12 04:01:17,936 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-12 04:01:17,937 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:17,937 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-12 04:01:17,937 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174477937"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:17,938 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174477937"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:17,938 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174477937"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:17,939 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-12 04:01:17,940 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 95c25e17def1a4a4a028fa2b6f6b5c57, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676174474334.95c25e17def1a4a4a028fa2b6f6b5c57.', STARTKEY => '', ENDKEY => ''}, {ENCODED => d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676174476405.d5a95b4a9e2a7dc12ff3a61b5a4b7ab1.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => e342c5ec8229f3d46866d60ac95bc270, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676174476405.e342c5ec8229f3d46866d60ac95bc270.', STARTKEY => '60', ENDKEY => ''}] 2023-02-12 04:01:17,940 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-12 04:01:17,940 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676174477940"}]},"ts":"9223372036854775807"} 2023-02-12 04:01:17,941 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-12 04:01:17,942 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:01:17,943 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 47 msec 2023-02-12 04:01:18,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-12 04:01:18,001 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-12 04:01:18,013 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=407 (was 402) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:55176 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:36366 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:47068 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:36350 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:60298 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:47066 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase20:40151-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:41010 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:60312 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:36372 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:36382 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:55008 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:41076 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x19f60b37-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:60282 [Waiting for operation #29] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:41386 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:47082 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-472446730_15 at /127.0.0.1:41068 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=857 (was 855) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=197 (was 197), ProcessCount=165 (was 165), AvailableMemoryMB=4079 (was 4133) 2023-02-12 04:01:18,026 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=407, OpenFileDescriptor=857, MaxFileDescriptor=60000, SystemLoadAverage=197, ProcessCount=165, AvailableMemoryMB=4079 2023-02-12 04:01:18,027 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:01:18,029 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-12 04:01:18,029 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-12 04:01:18,031 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:01:18,032 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 04:01:18,033 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 04:01:18,034 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:18,035 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42 empty. 2023-02-12 04:01:18,035 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:18,035 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-12 04:01:18,050 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-12 04:01:18,051 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 612e24e4f6a2dc8f5218f7fab3e82c42, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp 2023-02-12 04:01:18,061 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:18,061 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 612e24e4f6a2dc8f5218f7fab3e82c42, disabling compactions & flushes 2023-02-12 04:01:18,061 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:18,061 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:18,061 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. after waiting 0 ms 2023-02-12 04:01:18,061 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:18,061 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:18,061 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 612e24e4f6a2dc8f5218f7fab3e82c42: Waiting for close lock at 1676174478061Disabling compacts and flushes for region at 1676174478061Disabling writes for close at 1676174478061Writing region close event to WAL at 1676174478061Closed at 1676174478061 2023-02-12 04:01:18,063 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 04:01:18,064 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676174478064"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174478064"}]},"ts":"1676174478064"} 2023-02-12 04:01:18,065 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 04:01:18,066 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 04:01:18,066 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174478066"}]},"ts":"1676174478066"} 2023-02-12 04:01:18,067 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-12 04:01:18,071 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-12 04:01:18,072 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-12 04:01:18,072 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-12 04:01:18,072 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-12 04:01:18,072 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-12 04:01:18,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=612e24e4f6a2dc8f5218f7fab3e82c42, ASSIGN}] 2023-02-12 04:01:18,073 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=612e24e4f6a2dc8f5218f7fab3e82c42, ASSIGN 2023-02-12 04:01:18,074 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=612e24e4f6a2dc8f5218f7fab3e82c42, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,40151,1676174449139; forceNewPlan=false, retain=false 2023-02-12 04:01:18,224 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-12 04:01:18,227 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=612e24e4f6a2dc8f5218f7fab3e82c42, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:18,227 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676174478227"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174478227"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174478227"}]},"ts":"1676174478227"} 2023-02-12 04:01:18,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 612e24e4f6a2dc8f5218f7fab3e82c42, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:18,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:18,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 612e24e4f6a2dc8f5218f7fab3e82c42, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:01:18,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:01:18,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:18,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:18,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:18,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:18,384 INFO [StoreOpener-612e24e4f6a2dc8f5218f7fab3e82c42-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:18,385 DEBUG [StoreOpener-612e24e4f6a2dc8f5218f7fab3e82c42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42/cf1 2023-02-12 04:01:18,386 DEBUG [StoreOpener-612e24e4f6a2dc8f5218f7fab3e82c42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42/cf1 2023-02-12 04:01:18,386 INFO [StoreOpener-612e24e4f6a2dc8f5218f7fab3e82c42-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 612e24e4f6a2dc8f5218f7fab3e82c42 columnFamilyName cf1 2023-02-12 04:01:18,386 INFO [StoreOpener-612e24e4f6a2dc8f5218f7fab3e82c42-1] regionserver.HStore(310): Store=612e24e4f6a2dc8f5218f7fab3e82c42/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:18,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:18,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:18,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:18,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 04:01:18,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 612e24e4f6a2dc8f5218f7fab3e82c42; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68538747, jitterRate=0.021306917071342468}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 04:01:18,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 612e24e4f6a2dc8f5218f7fab3e82c42: Running coprocessor pre-open hook at 1676174478383Writing region info on filesystem at 1676174478383Initializing all the Stores at 1676174478384 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174478384Cleaning up temporary data from old regions at 1676174478388 (+4 ms)Cleaning up detritus from prior splits at 1676174478388Running coprocessor post-open hooks at 1676174478392 (+4 ms)Region opened successfully at 1676174478392 2023-02-12 04:01:18,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42., pid=87, masterSystemTime=1676174478380 2023-02-12 04:01:18,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:18,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:18,394 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=612e24e4f6a2dc8f5218f7fab3e82c42, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:18,394 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676174478394"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174478394"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174478394"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174478394"}]},"ts":"1676174478394"} 2023-02-12 04:01:18,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-12 04:01:18,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 612e24e4f6a2dc8f5218f7fab3e82c42, server=jenkins-hbase20.apache.org,40151,1676174449139 in 167 msec 2023-02-12 04:01:18,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-12 04:01:18,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=612e24e4f6a2dc8f5218f7fab3e82c42, ASSIGN in 325 msec 2023-02-12 04:01:18,399 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 04:01:18,399 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174478399"}]},"ts":"1676174478399"} 2023-02-12 04:01:18,400 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-12 04:01:18,401 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 04:01:18,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 371 msec 2023-02-12 04:01:18,540 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$12(2441): Client=jenkins//148.251.75.209 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-12 04:01:18,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:01:18,560 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-12 04:01:18,561 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-12 04:01:18,561 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-12 04:01:18,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-12 04:01:18,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=612e24e4f6a2dc8f5218f7fab3e82c42, REOPEN/MOVE}] 2023-02-12 04:01:18,577 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=612e24e4f6a2dc8f5218f7fab3e82c42, REOPEN/MOVE 2023-02-12 04:01:18,577 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=612e24e4f6a2dc8f5218f7fab3e82c42, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:18,577 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676174478577"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174478577"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174478577"}]},"ts":"1676174478577"} 2023-02-12 04:01:18,579 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 612e24e4f6a2dc8f5218f7fab3e82c42, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:18,731 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:18,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 612e24e4f6a2dc8f5218f7fab3e82c42, disabling compactions & flushes 2023-02-12 04:01:18,732 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:18,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:18,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. after waiting 0 ms 2023-02-12 04:01:18,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:18,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-12 04:01:18,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:01:18,736 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:18,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 612e24e4f6a2dc8f5218f7fab3e82c42: Waiting for close lock at 1676174478731Running coprocessor pre-close hooks at 1676174478731Disabling compacts and flushes for region at 1676174478731Disabling writes for close at 1676174478732 (+1 ms)Writing region close event to WAL at 1676174478733 (+1 ms)Running coprocessor post-close hooks at 1676174478736 (+3 ms)Closed at 1676174478736 2023-02-12 04:01:18,736 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 612e24e4f6a2dc8f5218f7fab3e82c42 to self. 2023-02-12 04:01:18,738 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:18,738 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=612e24e4f6a2dc8f5218f7fab3e82c42, regionState=CLOSED 2023-02-12 04:01:18,738 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676174478738"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174478738"}]},"ts":"1676174478738"} 2023-02-12 04:01:18,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-12 04:01:18,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 612e24e4f6a2dc8f5218f7fab3e82c42, server=jenkins-hbase20.apache.org,40151,1676174449139 in 160 msec 2023-02-12 04:01:18,741 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=612e24e4f6a2dc8f5218f7fab3e82c42, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase20.apache.org,40151,1676174449139; forceNewPlan=false, retain=true 2023-02-12 04:01:18,892 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-12 04:01:18,892 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=612e24e4f6a2dc8f5218f7fab3e82c42, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:18,892 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676174478892"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174478892"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174478892"}]},"ts":"1676174478892"} 2023-02-12 04:01:18,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 612e24e4f6a2dc8f5218f7fab3e82c42, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:01:19,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:19,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 612e24e4f6a2dc8f5218f7fab3e82c42, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:01:19,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:01:19,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:19,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:19,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:19,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:19,050 INFO [StoreOpener-612e24e4f6a2dc8f5218f7fab3e82c42-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:19,051 DEBUG [StoreOpener-612e24e4f6a2dc8f5218f7fab3e82c42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42/cf1 2023-02-12 04:01:19,051 DEBUG [StoreOpener-612e24e4f6a2dc8f5218f7fab3e82c42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42/cf1 2023-02-12 04:01:19,051 INFO [StoreOpener-612e24e4f6a2dc8f5218f7fab3e82c42-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 612e24e4f6a2dc8f5218f7fab3e82c42 columnFamilyName cf1 2023-02-12 04:01:19,052 INFO [StoreOpener-612e24e4f6a2dc8f5218f7fab3e82c42-1] regionserver.HStore(310): Store=612e24e4f6a2dc8f5218f7fab3e82c42/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:01:19,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:19,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:19,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:01:19,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 612e24e4f6a2dc8f5218f7fab3e82c42; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61021138, jitterRate=-0.09071418642997742}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 04:01:19,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 612e24e4f6a2dc8f5218f7fab3e82c42: Running coprocessor pre-open hook at 1676174479049Writing region info on filesystem at 1676174479049Initializing all the Stores at 1676174479050 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174479050Cleaning up temporary data from old regions at 1676174479054 (+4 ms)Cleaning up detritus from prior splits at 1676174479054Running coprocessor post-open hooks at 1676174479056 (+2 ms)Region opened successfully at 1676174479056 2023-02-12 04:01:19,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42., pid=92, masterSystemTime=1676174479045 2023-02-12 04:01:19,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:19,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:01:19,059 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=612e24e4f6a2dc8f5218f7fab3e82c42, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:01:19,059 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676174479059"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174479059"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174479059"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174479059"}]},"ts":"1676174479059"} 2023-02-12 04:01:19,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-12 04:01:19,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 612e24e4f6a2dc8f5218f7fab3e82c42, server=jenkins-hbase20.apache.org,40151,1676174449139 in 166 msec 2023-02-12 04:01:19,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-12 04:01:19,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=612e24e4f6a2dc8f5218f7fab3e82c42, REOPEN/MOVE in 486 msec 2023-02-12 04:01:19,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-12 04:01:19,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 500 msec 2023-02-12 04:01:19,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842, ASSIGN}] 2023-02-12 04:01:19,068 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842, ASSIGN 2023-02-12 04:01:19,069 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-12 04:01:19,219 DEBUG [jenkins-hbase20:41569] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-12 04:01:19,220 DEBUG [jenkins-hbase20:41569] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-12 04:01:19,220 DEBUG [jenkins-hbase20:41569] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-12 04:01:19,220 DEBUG [jenkins-hbase20:41569] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-12 04:01:19,220 DEBUG [jenkins-hbase20:41569] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-12 04:01:19,221 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=21357faa3e6d4ee08ffefe045d425842, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:01:19,222 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676174479221"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1676174479221"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676174479221"}]},"ts":"1676174479221"} 2023-02-12 04:01:19,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 21357faa3e6d4ee08ffefe045d425842, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:01:19,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676174478030_0001.21357faa3e6d4ee08ffefe045d425842. 2023-02-12 04:01:19,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 21357faa3e6d4ee08ffefe045d425842, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676174478030_0001.21357faa3e6d4ee08ffefe045d425842.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-12 04:01:19,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:01:19,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 21357faa3e6d4ee08ffefe045d425842 2023-02-12 04:01:19,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676174478030_0001.21357faa3e6d4ee08ffefe045d425842.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:01:19,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 21357faa3e6d4ee08ffefe045d425842 2023-02-12 04:01:19,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 21357faa3e6d4ee08ffefe045d425842 2023-02-12 04:01:19,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:19,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:19,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:19,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:19,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:19,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:19,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:20,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:20,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:20,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:20,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:20,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:20,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:20,592 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-12 04:01:20,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:20,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:20,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:20,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:21,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:21,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:21,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:21,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:21,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:21,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:21,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:21,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:21,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:21,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:22,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:22,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:22,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:22,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:22,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:22,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:22,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:22,712 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 04:01:22,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:22,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:22,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:23,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:23,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:23,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:23,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:23,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:23,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:23,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:23,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:23,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:23,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:24,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:24,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:24,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:24,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:24,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:24,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:24,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:24,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:24,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:24,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:25,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:25,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:25,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:25,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:25,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:25,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:25,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:25,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:25,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:25,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:26,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:26,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:26,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:26,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:26,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:26,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:26,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:26,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:26,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:26,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:27,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:27,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:27,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:27,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:27,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:27,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:27,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:27,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:27,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:28,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:28,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:28,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:28,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:28,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:28,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:28,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:28,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:28,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:28,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:29,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:29,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:29,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:29,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:29,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:29,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:29,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:29,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:29,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:29,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:30,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:30,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:30,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:30,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:30,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:30,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:30,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:30,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:30,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:30,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:31,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:31,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:31,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:31,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:31,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:31,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:31,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:31,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:31,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:31,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:32,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:32,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:32,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:32,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:32,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:32,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:32,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:32,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:32,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:32,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:33,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:33,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:33,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:33,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:33,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:33,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:33,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:33,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:33,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:33,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:34,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:34,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:34,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:34,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:34,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:34,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:34,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:34,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:34,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:34,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:35,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:35,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:35,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:35,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:35,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:35,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:35,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:35,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:35,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:35,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:36,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:36,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:36,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:36,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:36,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:36,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:36,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:36,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:36,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:36,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:37,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:37,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:37,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:37,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:37,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:37,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:37,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:37,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:37,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:37,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:38,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:38,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:38,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:38,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:38,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:38,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:38,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:38,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:38,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:38,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:39,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:39,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:39,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:39,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:39,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:39,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:39,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:39,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:39,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:39,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:40,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:40,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:40,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:40,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:40,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:40,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:40,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:40,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:40,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:40,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:41,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:41,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:41,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:41,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:41,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:41,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:41,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:41,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:41,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:41,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:42,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:42,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:42,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:42,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:42,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:42,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:42,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:42,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:42,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:42,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:43,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:43,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:43,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:43,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:43,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:43,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:43,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:43,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:43,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:43,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:44,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:44,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:44,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:44,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:44,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:44,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:44,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:44,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:44,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:44,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:45,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:45,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:45,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:45,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:45,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:45,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:45,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:45,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:45,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:45,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:46,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:46,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:46,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:46,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:46,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:46,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:46,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:46,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:46,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:46,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:47,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:47,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:47,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:47,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:47,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:47,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:47,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:47,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:47,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:47,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:48,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:48,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:48,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:48,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:48,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:48,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:48,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:48,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:48,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:48,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:49,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:49,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:49,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:49,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:49,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:49,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:49,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:49,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:49,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:49,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:50,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:50,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:50,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:50,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:50,458 DEBUG [regionserver/jenkins-hbase20:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-12 04:01:50,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:50,475 DEBUG [regionserver/jenkins-hbase20:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-12 04:01:50,475 DEBUG [regionserver/jenkins-hbase20:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-12 04:01:50,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:50,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:50,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:50,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:50,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:51,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:51,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:51,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:51,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:51,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:51,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:51,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:51,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:51,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:51,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:52,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:52,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:52,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:52,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:52,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:52,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:52,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:52,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:52,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:52,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:53,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:53,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:53,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:53,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:53,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:53,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:53,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:53,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:53,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:53,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:54,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:54,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:54,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:54,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:54,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:54,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:54,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:54,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:54,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:54,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:55,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:55,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:55,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:55,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:55,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:55,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:55,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:55,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:55,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:55,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:56,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:56,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:56,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:56,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:56,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:56,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:56,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:56,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:56,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:56,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:57,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:57,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:57,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:57,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:57,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:57,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:57,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:57,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:57,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:57,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:58,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:58,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:58,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:58,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:58,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:58,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:58,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:58,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:58,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:58,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:59,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:59,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:59,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:59,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:59,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:59,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:59,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:59,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:59,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:01:59,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:00,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:00,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:00,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:00,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:00,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:00,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:00,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:00,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:00,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:00,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:01,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:01,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:01,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:01,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:01,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:01,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:01,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:01,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:01,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:02,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:02,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:02,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:02,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:02,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:02,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:02,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:02,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:02,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:02,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:03,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:03,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:03,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:03,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:03,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:03,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:03,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:03,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:03,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:03,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:04,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:04,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:04,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:04,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:04,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:04,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:04,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:04,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:04,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:04,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:05,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:05,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:05,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:05,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:05,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:05,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:05,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:05,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:05,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:05,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:06,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:06,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:06,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:06,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:06,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:06,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:06,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:06,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:06,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:06,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:07,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:07,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:07,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:07,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:07,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:07,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:07,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:07,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:07,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:07,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:08,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:08,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:08,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:08,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:08,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:08,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:08,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:08,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:08,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:08,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:09,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:09,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:09,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:09,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:09,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:09,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:09,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:09,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:09,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:09,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:10,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:10,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:10,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:10,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:10,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:10,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:10,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:10,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:10,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:10,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:11,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:11,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:11,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:11,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:11,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:11,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:11,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:11,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:11,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:11,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:12,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:12,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:12,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:12,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:12,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:12,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:12,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:12,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:12,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:12,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:13,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:13,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:13,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:13,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:13,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:13,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:13,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:13,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:13,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:13,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:14,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:14,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:14,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:14,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:14,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:14,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:14,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:14,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:14,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:14,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:15,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:15,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:15,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:15,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:15,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:15,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:15,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:15,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:15,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:15,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:16,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:16,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:16,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:16,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:16,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:16,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:16,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:16,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:16,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:16,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:17,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:17,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:17,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:17,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:17,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:17,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:17,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:17,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:17,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:17,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:18,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:18,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:18,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:18,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:18,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:18,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:18,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:18,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-12 04:02:18,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:18,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-12 04:02:18,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:18,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:18,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-12 04:02:19,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:19,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:19,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:19,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-12 04:02:19,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:19,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:19,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:19,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:19,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:19,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-12 04:02:19,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:19,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:20,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:20,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:20,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:20,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:20,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:20,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:20,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:20,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:20,775 DEBUG [Listener at localhost.localdomain/35207] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:20,776 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:20,777 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:20,777 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:20,777 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:20,779 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 612e24e4f6a2dc8f5218f7fab3e82c42, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:02:20,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 exec-time=8 msec 2023-02-12 04:02:20,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:20,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-12 04:02:21,061 INFO [StoreOpener-21357faa3e6d4ee08ffefe045d425842-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 21357faa3e6d4ee08ffefe045d425842 2023-02-12 04:02:21,068 DEBUG [StoreOpener-21357faa3e6d4ee08ffefe045d425842-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42/cf1 2023-02-12 04:02:21,068 DEBUG [StoreOpener-21357faa3e6d4ee08ffefe045d425842-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42/cf1 2023-02-12 04:02:21,069 INFO [StoreOpener-21357faa3e6d4ee08ffefe045d425842-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 21357faa3e6d4ee08ffefe045d425842 columnFamilyName cf1 2023-02-12 04:02:21,071 INFO [StoreOpener-21357faa3e6d4ee08ffefe045d425842-1] regionserver.HStore(310): Store=21357faa3e6d4ee08ffefe045d425842/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:21,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 21357faa3e6d4ee08ffefe045d425842; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69606706, jitterRate=0.03722074627876282}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 04:02:21,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 21357faa3e6d4ee08ffefe045d425842: Running coprocessor pre-open hook at 1676174479380Initializing all the Stores at 1676174541060 (+61680 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174541060Running coprocessor post-open hooks at 1676174541075 (+15 ms)Region opened successfully at 1676174541075 2023-02-12 04:02:21,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676174478030_0001.21357faa3e6d4ee08ffefe045d425842., pid=94, masterSystemTime=1676174479376 2023-02-12 04:02:21,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676174478030_0001.21357faa3e6d4ee08ffefe045d425842. 2023-02-12 04:02:21,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676174478030_0001.21357faa3e6d4ee08ffefe045d425842. 2023-02-12 04:02:21,077 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=21357faa3e6d4ee08ffefe045d425842, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:21,077 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676174541077"},{"qualifier":"server_0001","vlen":32,"tag":[],"timestamp":"1676174541077"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676174541077"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676174541077"}]},"ts":"1676174541077"} 2023-02-12 04:02:21,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-12 04:02:21,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 21357faa3e6d4ee08ffefe045d425842, server=jenkins-hbase20.apache.org,43977,1676174449184 in 1 mins, 1.855 sec 2023-02-12 04:02:21,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-12 04:02:21,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842, ASSIGN in 1 mins, 2.013 sec 2023-02-12 04:02:21,082 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.538 sec 2023-02-12 04:02:21,082 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676174478538","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62544,"client":"148.251.75.209:55886","queuetimems":0,"class":"HMaster"} 2023-02-12 04:02:21,083 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 603, skipping over this response of 3 bytes 2023-02-12 04:02:23,027 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up 2023-02-12 04:02:23,030 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:02:23,034 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:23,035 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:23,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:23,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-12 04:02:23,041 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174543041"}]},"ts":"1676174543041"} 2023-02-12 04:02:23,043 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-12 04:02:23,045 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-12 04:02:23,046 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=612e24e4f6a2dc8f5218f7fab3e82c42, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842, UNASSIGN}] 2023-02-12 04:02:23,047 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842, UNASSIGN 2023-02-12 04:02:23,047 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=21357faa3e6d4ee08ffefe045d425842, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:23,048 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676174543047"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1676174543047"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676174543047"}]},"ts":"1676174543047"} 2023-02-12 04:02:23,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 21357faa3e6d4ee08ffefe045d425842, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:02:23,051 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=612e24e4f6a2dc8f5218f7fab3e82c42, UNASSIGN 2023-02-12 04:02:23,052 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=612e24e4f6a2dc8f5218f7fab3e82c42, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:23,052 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676174543052"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174543052"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174543052"}]},"ts":"1676174543052"} 2023-02-12 04:02:23,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 612e24e4f6a2dc8f5218f7fab3e82c42, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:02:23,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-12 04:02:23,201 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 21357faa3e6d4ee08ffefe045d425842 2023-02-12 04:02:23,202 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 21357faa3e6d4ee08ffefe045d425842, disabling compactions & flushes 2023-02-12 04:02:23,203 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676174478030_0001.21357faa3e6d4ee08ffefe045d425842. 2023-02-12 04:02:23,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676174478030_0001.21357faa3e6d4ee08ffefe045d425842. 2023-02-12 04:02:23,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676174478030_0001.21357faa3e6d4ee08ffefe045d425842. after waiting 0 ms 2023-02-12 04:02:23,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676174478030_0001.21357faa3e6d4ee08ffefe045d425842. 2023-02-12 04:02:23,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:23,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676174478030_0001.21357faa3e6d4ee08ffefe045d425842. 2023-02-12 04:02:23,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 21357faa3e6d4ee08ffefe045d425842: Waiting for close lock at 1676174543202Running coprocessor pre-close hooks at 1676174543202Disabling compacts and flushes for region at 1676174543202Disabling writes for close at 1676174543203 (+1 ms)Writing region close event to WAL at 1676174543203Running coprocessor post-close hooks at 1676174543203Closed at 1676174543204 (+1 ms) 2023-02-12 04:02:23,206 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:02:23,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 612e24e4f6a2dc8f5218f7fab3e82c42, disabling compactions & flushes 2023-02-12 04:02:23,207 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:02:23,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:02:23,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. after waiting 0 ms 2023-02-12 04:02:23,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:02:23,207 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 21357faa3e6d4ee08ffefe045d425842 2023-02-12 04:02:23,208 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=21357faa3e6d4ee08ffefe045d425842, regionState=CLOSED 2023-02-12 04:02:23,209 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676174543208"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676174543208"}]},"ts":"1676174543208"} 2023-02-12 04:02:23,215 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-12 04:02:23,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-12 04:02:23,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 21357faa3e6d4ee08ffefe045d425842, server=jenkins-hbase20.apache.org,43977,1676174449184 in 164 msec 2023-02-12 04:02:23,216 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:23,217 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42. 2023-02-12 04:02:23,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 612e24e4f6a2dc8f5218f7fab3e82c42: Waiting for close lock at 1676174543206Running coprocessor pre-close hooks at 1676174543206Disabling compacts and flushes for region at 1676174543206Disabling writes for close at 1676174543207 (+1 ms)Writing region close event to WAL at 1676174543208 (+1 ms)Running coprocessor post-close hooks at 1676174543216 (+8 ms)Closed at 1676174543217 (+1 ms) 2023-02-12 04:02:23,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842, UNASSIGN in 169 msec 2023-02-12 04:02:23,218 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:02:23,218 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=612e24e4f6a2dc8f5218f7fab3e82c42, regionState=CLOSED 2023-02-12 04:02:23,218 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676174543218"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174543218"}]},"ts":"1676174543218"} 2023-02-12 04:02:23,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-12 04:02:23,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 612e24e4f6a2dc8f5218f7fab3e82c42, server=jenkins-hbase20.apache.org,40151,1676174449139 in 167 msec 2023-02-12 04:02:23,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-12 04:02:23,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=612e24e4f6a2dc8f5218f7fab3e82c42, UNASSIGN in 175 msec 2023-02-12 04:02:23,223 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174543223"}]},"ts":"1676174543223"} 2023-02-12 04:02:23,224 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-12 04:02:23,225 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-12 04:02:23,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 191 msec 2023-02-12 04:02:23,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-12 04:02:23,345 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-12 04:02:23,347 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:23,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:23,352 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:23,353 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:23,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-12 04:02:23,359 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:02:23,361 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42/recovered.edits] 2023-02-12 04:02:23,368 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42/recovered.edits/7.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42/recovered.edits/7.seqid 2023-02-12 04:02:23,369 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/612e24e4f6a2dc8f5218f7fab3e82c42 2023-02-12 04:02:23,369 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-12 04:02:23,372 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:23,375 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-12 04:02:23,378 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-12 04:02:23,379 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:23,379 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-12 04:02:23,379 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174543379"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:23,379 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676174478030_0001.21357faa3e6d4ee08ffefe045d425842.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174543379"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:23,382 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-12 04:02:23,382 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 612e24e4f6a2dc8f5218f7fab3e82c42, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676174478030.612e24e4f6a2dc8f5218f7fab3e82c42.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 21357faa3e6d4ee08ffefe045d425842, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676174478030_0001.21357faa3e6d4ee08ffefe045d425842.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-12 04:02:23,382 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-12 04:02:23,382 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676174543382"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:23,385 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-12 04:02:23,387 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:23,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 40 msec 2023-02-12 04:02:23,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-12 04:02:23,457 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-12 04:02:23,482 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=341 (was 407), OpenFileDescriptor=649 (was 857), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=127 (was 197), ProcessCount=163 (was 165), AvailableMemoryMB=4895 (was 4079) - AvailableMemoryMB LEAK? - 2023-02-12 04:02:23,494 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=342, OpenFileDescriptor=649, MaxFileDescriptor=60000, SystemLoadAverage=127, ProcessCount=163, AvailableMemoryMB=4894 2023-02-12 04:02:23,494 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:02:23,496 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-12 04:02:23,497 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-12 04:02:23,498 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:23,500 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 04:02:23,500 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 04:02:23,502 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:23,502 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf empty. 2023-02-12 04:02:23,503 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:23,503 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-12 04:02:23,514 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-12 04:02:23,516 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 35ea055f2d9a2a9b0e631baf23e56ecf, NAME => 'testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp 2023-02-12 04:02:23,525 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:23,525 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 35ea055f2d9a2a9b0e631baf23e56ecf, disabling compactions & flushes 2023-02-12 04:02:23,525 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:23,525 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:23,525 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. after waiting 0 ms 2023-02-12 04:02:23,525 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:23,525 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:23,525 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 35ea055f2d9a2a9b0e631baf23e56ecf: Waiting for close lock at 1676174543525Disabling compacts and flushes for region at 1676174543525Disabling writes for close at 1676174543525Writing region close event to WAL at 1676174543525Closed at 1676174543525 2023-02-12 04:02:23,527 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 04:02:23,528 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676174543528"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174543528"}]},"ts":"1676174543528"} 2023-02-12 04:02:23,530 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 04:02:23,531 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 04:02:23,531 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174543531"}]},"ts":"1676174543531"} 2023-02-12 04:02:23,533 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-12 04:02:23,536 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-12 04:02:23,537 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-12 04:02:23,538 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-12 04:02:23,538 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-12 04:02:23,538 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-12 04:02:23,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=35ea055f2d9a2a9b0e631baf23e56ecf, ASSIGN}] 2023-02-12 04:02:23,540 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=35ea055f2d9a2a9b0e631baf23e56ecf, ASSIGN 2023-02-12 04:02:23,541 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=35ea055f2d9a2a9b0e631baf23e56ecf, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,43977,1676174449184; forceNewPlan=false, retain=false 2023-02-12 04:02:23,692 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-12 04:02:23,692 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=35ea055f2d9a2a9b0e631baf23e56ecf, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:23,693 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676174543692"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174543692"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174543692"}]},"ts":"1676174543692"} 2023-02-12 04:02:23,694 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 35ea055f2d9a2a9b0e631baf23e56ecf, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:02:23,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:23,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 35ea055f2d9a2a9b0e631baf23e56ecf, NAME => 'testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:02:23,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:02:23,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:23,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:23,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:23,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:23,855 INFO [StoreOpener-35ea055f2d9a2a9b0e631baf23e56ecf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:23,858 DEBUG [StoreOpener-35ea055f2d9a2a9b0e631baf23e56ecf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1 2023-02-12 04:02:23,858 DEBUG [StoreOpener-35ea055f2d9a2a9b0e631baf23e56ecf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1 2023-02-12 04:02:23,858 INFO [StoreOpener-35ea055f2d9a2a9b0e631baf23e56ecf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 35ea055f2d9a2a9b0e631baf23e56ecf columnFamilyName cf1 2023-02-12 04:02:23,859 INFO [StoreOpener-35ea055f2d9a2a9b0e631baf23e56ecf-1] regionserver.HStore(310): Store=35ea055f2d9a2a9b0e631baf23e56ecf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:23,859 INFO [StoreOpener-35ea055f2d9a2a9b0e631baf23e56ecf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:23,861 DEBUG [StoreOpener-35ea055f2d9a2a9b0e631baf23e56ecf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2 2023-02-12 04:02:23,861 DEBUG [StoreOpener-35ea055f2d9a2a9b0e631baf23e56ecf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2 2023-02-12 04:02:23,861 INFO [StoreOpener-35ea055f2d9a2a9b0e631baf23e56ecf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 35ea055f2d9a2a9b0e631baf23e56ecf columnFamilyName cf2 2023-02-12 04:02:23,862 INFO [StoreOpener-35ea055f2d9a2a9b0e631baf23e56ecf-1] regionserver.HStore(310): Store=35ea055f2d9a2a9b0e631baf23e56ecf/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:23,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:23,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:23,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:02:23,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:23,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 04:02:23,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 35ea055f2d9a2a9b0e631baf23e56ecf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73687435, jitterRate=0.09802834689617157}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:02:23,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 35ea055f2d9a2a9b0e631baf23e56ecf: Running coprocessor pre-open hook at 1676174543853Writing region info on filesystem at 1676174543853Initializing all the Stores at 1676174543855 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174543855Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174543855Cleaning up temporary data from old regions at 1676174543863 (+8 ms)Cleaning up detritus from prior splits at 1676174543863Running coprocessor post-open hooks at 1676174543867 (+4 ms)Region opened successfully at 1676174543867 2023-02-12 04:02:23,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf., pid=104, masterSystemTime=1676174543846 2023-02-12 04:02:23,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:23,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:23,869 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=35ea055f2d9a2a9b0e631baf23e56ecf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:23,869 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676174543869"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174543869"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174543869"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174543869"}]},"ts":"1676174543869"} 2023-02-12 04:02:23,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-12 04:02:23,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 35ea055f2d9a2a9b0e631baf23e56ecf, server=jenkins-hbase20.apache.org,43977,1676174449184 in 176 msec 2023-02-12 04:02:23,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-12 04:02:23,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=35ea055f2d9a2a9b0e631baf23e56ecf, ASSIGN in 334 msec 2023-02-12 04:02:23,874 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 04:02:23,874 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174543874"}]},"ts":"1676174543874"} 2023-02-12 04:02:23,875 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-12 04:02:23,876 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 04:02:23,877 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 379 msec 2023-02-12 04:02:24,006 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:24,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-12 04:02:24,008 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-12 04:02:24,008 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,008 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-12 04:02:24,008 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-12 04:02:24,009 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,009 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,010 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,010 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,010 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,010 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,010 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,010 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:24,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,011 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,011 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,011 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,011 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,011 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,011 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,011 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,011 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,011 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,011 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,011 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,011 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,011 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,011 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,011 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,011 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,011 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,011 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,011 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,011 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,012 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:24,012 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:24,012 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:24,012 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. started... 2023-02-12 04:02:24,012 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 35ea055f2d9a2a9b0e631baf23e56ecf 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-12 04:02:24,013 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,013 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,013 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,013 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,013 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,014 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,022 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/6944369283534277b8d50ed63799e024 2023-02-12 04:02:24,038 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/22c09705f1b94702990803ec235e0a1e 2023-02-12 04:02:24,042 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/6944369283534277b8d50ed63799e024 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/6944369283534277b8d50ed63799e024 2023-02-12 04:02:24,046 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/6944369283534277b8d50ed63799e024, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:02:24,046 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/22c09705f1b94702990803ec235e0a1e as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/22c09705f1b94702990803ec235e0a1e 2023-02-12 04:02:24,054 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/22c09705f1b94702990803ec235e0a1e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:02:24,054 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 35ea055f2d9a2a9b0e631baf23e56ecf in 42ms, sequenceid=6, compaction requested=false 2023-02-12 04:02:24,054 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-12 04:02:24,055 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 35ea055f2d9a2a9b0e631baf23e56ecf: 2023-02-12 04:02:24,055 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:24,055 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:24,055 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:24,055 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,055 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,055 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,056 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,056 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,057 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,057 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-12 04:02:24,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@379487c[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:24,059 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-12 04:02:24,059 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,059 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,059 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,059 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,059 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,060 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:24,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,059 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,060 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,060 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,060 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,060 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:24,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,060 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:24,060 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,060 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,060 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:24,061 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,061 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,061 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,061 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,061 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,061 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,061 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,061 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,061 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,061 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:02:24,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-12 04:02:24,064 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-12 04:02:24,064 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:24,064 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:24,064 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-12 04:02:24,064 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:24,065 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,065 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,065 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,065 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,065 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,065 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,065 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,065 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,066 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,066 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,067 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,068 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,068 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,074 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,074 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,074 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,075 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,074 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,074 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,075 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,075 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-12 04:02:24,075 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:24,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,075 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,075 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,075 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:24,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,075 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,075 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:24,075 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,076 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,076 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,076 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,076 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,076 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,076 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,076 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,176 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:24,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:24,191 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:24,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:24,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-12 04:02:24,193 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-12 04:02:24,193 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,193 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-12 04:02:24,193 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-12 04:02:24,194 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,194 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,196 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,196 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,196 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,196 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,196 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,196 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:24,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,197 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,198 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,198 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,198 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,198 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,198 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,198 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,198 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,199 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,198 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,199 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,199 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,199 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,199 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,199 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,199 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,199 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,199 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,199 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,199 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,199 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,199 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,199 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,199 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,200 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,200 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,200 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,201 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,201 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,201 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,201 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,201 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:24,201 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:24,201 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:24,201 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. started... 2023-02-12 04:02:24,201 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 35ea055f2d9a2a9b0e631baf23e56ecf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:24,214 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/e1d6ae305cd14079934505f506d79067 2023-02-12 04:02:24,228 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/91d4192175ad4cd5bc0f0184f9e838f7 2023-02-12 04:02:24,234 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/e1d6ae305cd14079934505f506d79067 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/e1d6ae305cd14079934505f506d79067 2023-02-12 04:02:24,240 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/e1d6ae305cd14079934505f506d79067, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:02:24,241 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/91d4192175ad4cd5bc0f0184f9e838f7 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/91d4192175ad4cd5bc0f0184f9e838f7 2023-02-12 04:02:24,246 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/91d4192175ad4cd5bc0f0184f9e838f7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:02:24,247 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 35ea055f2d9a2a9b0e631baf23e56ecf in 46ms, sequenceid=19, compaction requested=false 2023-02-12 04:02:24,247 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 35ea055f2d9a2a9b0e631baf23e56ecf: 2023-02-12 04:02:24,247 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:24,247 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:24,247 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:24,247 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,247 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,247 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,248 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,248 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,248 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,248 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,250 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-12 04:02:24,250 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2cad0381[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:24,250 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-12 04:02:24,250 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,251 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,251 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,251 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,251 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,251 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,252 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,252 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:24,251 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:24,252 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,252 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,252 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,252 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,252 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,252 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,252 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,252 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,253 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,253 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:02:24,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-12 04:02:24,255 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-12 04:02:24,255 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:24,255 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:24,255 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-12 04:02:24,255 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:24,256 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,256 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,256 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,256 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,256 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,256 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,256 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,257 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,257 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,257 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,259 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,259 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,259 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,261 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-12 04:02:24,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,261 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,261 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:24,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,262 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:24,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,261 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,262 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,262 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,262 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,362 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:24,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:24,373 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:24,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:24,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-12 04:02:24,374 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-12 04:02:24,374 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,374 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-12 04:02:24,374 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-12 04:02:24,375 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,375 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,375 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,376 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,376 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,376 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,376 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,376 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:24,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,377 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,377 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,377 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,377 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,377 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,377 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,377 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,377 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,377 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,377 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:24,377 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:24,377 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. started... 2023-02-12 04:02:24,378 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 35ea055f2d9a2a9b0e631baf23e56ecf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:24,378 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,378 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,378 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,378 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,378 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,379 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,388 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/ce28a1662dc3498498e2a0a89ca465b5 2023-02-12 04:02:24,398 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/ece11c370d624237957f1278a5118710 2023-02-12 04:02:24,402 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/ce28a1662dc3498498e2a0a89ca465b5 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/ce28a1662dc3498498e2a0a89ca465b5 2023-02-12 04:02:24,405 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/ce28a1662dc3498498e2a0a89ca465b5, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:02:24,405 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/ece11c370d624237957f1278a5118710 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/ece11c370d624237957f1278a5118710 2023-02-12 04:02:24,409 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/ece11c370d624237957f1278a5118710, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:02:24,409 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 35ea055f2d9a2a9b0e631baf23e56ecf in 31ms, sequenceid=32, compaction requested=true 2023-02-12 04:02:24,410 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 35ea055f2d9a2a9b0e631baf23e56ecf: 2023-02-12 04:02:24,410 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:24,410 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:24,410 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:24,410 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,410 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,410 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,411 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,411 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,411 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,411 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,413 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-12 04:02:24,413 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3be75136[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:24,413 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-12 04:02:24,413 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,413 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,413 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,413 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,414 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,414 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,414 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:24,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,414 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,414 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:24,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,414 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,414 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,414 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:24,414 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,414 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,414 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:24,414 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,414 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,414 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,415 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,415 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,415 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,415 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,415 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,415 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,415 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:02:24,417 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-12 04:02:24,417 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-12 04:02:24,417 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:24,417 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:24,417 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-12 04:02:24,417 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:24,418 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,418 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,418 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,418 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,418 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,418 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,418 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,419 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,419 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,419 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,420 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,421 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,421 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,423 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,423 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,423 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,423 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,423 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,423 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,423 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,423 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,423 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,423 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,423 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-12 04:02:24,424 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:24,423 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,424 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:24,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,424 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,424 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:24,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,424 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,424 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,424 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,424 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,424 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,424 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,524 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:24,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:24,534 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:24,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:24,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-12 04:02:24,536 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-12 04:02:24,536 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,536 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-12 04:02:24,536 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-12 04:02:24,536 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,536 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,537 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,537 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,537 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,537 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,537 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,537 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:24,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,538 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,538 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,538 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,538 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,538 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,538 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,538 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,538 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,538 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,538 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,539 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:24,539 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:24,539 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:24,539 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,539 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. started... 2023-02-12 04:02:24,539 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,539 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,539 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,539 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 35ea055f2d9a2a9b0e631baf23e56ecf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:24,539 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,539 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,539 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,539 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,539 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,539 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,540 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,540 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,540 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,545 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/b3ac813e0f1f469ea546b6ebea87cc03 2023-02-12 04:02:24,554 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/e6ae8b09cef34e0eb239789b5b0ea1f3 2023-02-12 04:02:24,558 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/b3ac813e0f1f469ea546b6ebea87cc03 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/b3ac813e0f1f469ea546b6ebea87cc03 2023-02-12 04:02:24,560 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/b3ac813e0f1f469ea546b6ebea87cc03, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:02:24,561 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/e6ae8b09cef34e0eb239789b5b0ea1f3 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/e6ae8b09cef34e0eb239789b5b0ea1f3 2023-02-12 04:02:24,564 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/e6ae8b09cef34e0eb239789b5b0ea1f3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:02:24,565 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 35ea055f2d9a2a9b0e631baf23e56ecf in 26ms, sequenceid=45, compaction requested=true 2023-02-12 04:02:24,565 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 35ea055f2d9a2a9b0e631baf23e56ecf: 2023-02-12 04:02:24,565 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:24,565 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:24,565 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:24,565 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,565 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,565 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,566 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,566 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,566 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,566 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,567 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-12 04:02:24,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45440e32[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:24,568 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-12 04:02:24,568 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,568 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,568 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,568 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,568 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,568 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,568 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:24,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,568 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,568 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,569 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,569 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,569 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,569 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:24,569 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:24,569 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:24,569 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,569 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,569 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,570 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,570 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,570 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,570 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,570 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,570 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,570 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:02:24,572 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-12 04:02:24,572 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-12 04:02:24,572 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:24,572 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:24,572 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-12 04:02:24,572 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:24,573 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,573 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,573 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,573 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,573 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,573 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,573 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,573 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,574 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,574 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,575 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,575 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,575 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,577 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,577 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,577 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,577 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,577 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,577 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,577 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,577 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,577 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-12 04:02:24,577 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,577 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,577 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,577 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:24,577 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,578 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:24,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,678 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:24,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:24,695 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:24,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:24,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-12 04:02:24,697 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-12 04:02:24,698 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,698 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-12 04:02:24,698 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-12 04:02:24,698 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,698 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,699 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,699 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,700 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,699 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,700 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,700 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:24,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,701 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,702 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,702 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,702 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,702 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,702 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,702 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,702 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,702 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,702 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,703 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,703 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,703 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,703 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,703 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:24,703 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:24,703 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:24,703 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. started... 2023-02-12 04:02:24,703 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 35ea055f2d9a2a9b0e631baf23e56ecf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:24,704 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,704 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,704 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,704 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,704 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,704 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,714 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/e84c73798c58441c91d004195629c650 2023-02-12 04:02:24,730 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/0336854937624d46bf9090cd9dbced81 2023-02-12 04:02:24,735 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/e84c73798c58441c91d004195629c650 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/e84c73798c58441c91d004195629c650 2023-02-12 04:02:24,738 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/e84c73798c58441c91d004195629c650, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:02:24,739 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/0336854937624d46bf9090cd9dbced81 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/0336854937624d46bf9090cd9dbced81 2023-02-12 04:02:24,743 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/0336854937624d46bf9090cd9dbced81, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:02:24,744 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 35ea055f2d9a2a9b0e631baf23e56ecf in 41ms, sequenceid=58, compaction requested=true 2023-02-12 04:02:24,744 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 35ea055f2d9a2a9b0e631baf23e56ecf: 2023-02-12 04:02:24,744 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:24,744 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:24,744 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:24,744 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,744 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,744 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,746 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,746 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,746 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,746 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-12 04:02:24,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@cb7944e[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:24,749 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-12 04:02:24,749 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,754 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,754 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,754 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,754 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,754 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,754 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,754 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:24,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,754 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:24,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,754 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,754 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,754 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,755 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:24,754 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,755 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:24,755 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,755 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,755 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,756 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,756 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,756 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,756 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,756 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,756 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,756 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:02:24,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-12 04:02:24,761 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-12 04:02:24,761 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:24,761 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:24,761 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-12 04:02:24,761 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:24,763 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,763 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,763 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,763 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,763 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,763 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,763 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,764 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,764 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,765 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,767 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,768 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,768 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,783 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,783 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,783 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,783 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,783 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,784 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,783 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,784 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-12 04:02:24,784 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:24,784 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,783 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,783 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,785 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,784 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,785 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,785 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:24,786 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:24,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,785 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,786 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,786 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,786 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,786 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,886 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:24,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:24,909 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:24,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:24,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-12 04:02:24,910 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-12 04:02:24,910 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,910 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-12 04:02:24,910 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-12 04:02:24,911 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,911 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,912 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,912 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,912 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,912 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,912 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,913 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:24,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,913 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,913 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,913 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,913 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,914 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:24,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,914 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,914 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-12 04:02:24,914 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,914 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,914 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,914 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-12 04:02:24,914 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,914 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-12 04:02:24,914 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,914 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,914 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,914 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-12 04:02:24,914 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,914 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,914 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:24,914 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,914 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:24,914 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:24,914 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:24,914 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,914 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-12 04:02:24,914 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. started... 2023-02-12 04:02:24,915 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,915 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,915 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 35ea055f2d9a2a9b0e631baf23e56ecf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:24,915 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,916 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,916 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,916 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,916 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,916 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,925 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/1e41abd4aa6642df82bd045d0dfa5ef3 2023-02-12 04:02:24,940 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/7bfc88b3c18c4044920fa846071ee349 2023-02-12 04:02:24,945 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/1e41abd4aa6642df82bd045d0dfa5ef3 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/1e41abd4aa6642df82bd045d0dfa5ef3 2023-02-12 04:02:24,950 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/1e41abd4aa6642df82bd045d0dfa5ef3, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:02:24,951 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/7bfc88b3c18c4044920fa846071ee349 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/7bfc88b3c18c4044920fa846071ee349 2023-02-12 04:02:24,956 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/7bfc88b3c18c4044920fa846071ee349, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:02:24,957 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 35ea055f2d9a2a9b0e631baf23e56ecf in 42ms, sequenceid=71, compaction requested=true 2023-02-12 04:02:24,957 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 35ea055f2d9a2a9b0e631baf23e56ecf: 2023-02-12 04:02:24,958 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:24,958 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:24,958 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:24,958 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,958 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-12 04:02:24,958 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-12 04:02:24,959 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,959 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,960 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,960 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:24,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-12 04:02:24,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48423f52[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:24,962 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-12 04:02:24,963 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,963 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,963 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,963 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,964 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,964 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,964 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:24,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,964 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,964 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:24,964 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,964 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-12 04:02:24,964 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,964 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-12 04:02:24,964 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:24,964 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:24,965 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,965 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,965 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,965 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,965 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,965 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,965 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-12 04:02:24,965 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,966 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,966 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-12 04:02:24,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:02:24,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-12 04:02:24,969 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-12 04:02:24,969 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:24,969 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:24,970 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-12 04:02:24,970 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:24,971 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,971 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,971 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,971 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,971 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,971 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,971 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,972 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:24,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:24,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:24,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,972 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:24,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,973 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:24,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,975 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:24,975 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-12 04:02:24,976 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,986 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,987 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,987 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,987 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,986 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:24,987 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:24,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,987 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,987 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-12 04:02:24,987 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:24,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,987 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:24,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:24,987 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:24,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,988 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:24,988 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,988 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:24,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:24,988 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,989 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-12 04:02:24,989 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:24,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:24,989 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,989 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:24,989 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:24,989 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-12 04:02:24,989 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-12 04:02:25,089 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:25,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:25,106 INFO [Listener at localhost.localdomain/35207] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:25,107 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43977] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/e1d6ae305cd14079934505f506d79067 because midkey is the same as first or last row 2023-02-12 04:02:25,108 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43977] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/91d4192175ad4cd5bc0f0184f9e838f7 because midkey is the same as first or last row 2023-02-12 04:02:25,108 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43977] regionserver.HRegion(2744): Flushing 35ea055f2d9a2a9b0e631baf23e56ecf 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-12 04:02:25,120 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43977] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/3d6bbe08df704aa7a83678454b067d05 2023-02-12 04:02:25,135 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43977] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/81fe6bd0846e4c58a1ec3410fd033a24 2023-02-12 04:02:25,140 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf1/3d6bbe08df704aa7a83678454b067d05 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/3d6bbe08df704aa7a83678454b067d05 2023-02-12 04:02:25,148 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43977] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/3d6bbe08df704aa7a83678454b067d05, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:02:25,149 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/.tmp/cf2/81fe6bd0846e4c58a1ec3410fd033a24 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/81fe6bd0846e4c58a1ec3410fd033a24 2023-02-12 04:02:25,156 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43977] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/81fe6bd0846e4c58a1ec3410fd033a24, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:02:25,157 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43977] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 35ea055f2d9a2a9b0e631baf23e56ecf in 49ms, sequenceid=82, compaction requested=true 2023-02-12 04:02:25,158 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43977] regionserver.HRegion(2445): Flush status journal for 35ea055f2d9a2a9b0e631baf23e56ecf: 2023-02-12 04:02:25,158 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43977] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/e1d6ae305cd14079934505f506d79067 because midkey is the same as first or last row 2023-02-12 04:02:25,158 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43977] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/91d4192175ad4cd5bc0f0184f9e838f7 because midkey is the same as first or last row 2023-02-12 04:02:25,158 DEBUG [Listener at localhost.localdomain/35207] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:25,158 DEBUG [Listener at localhost.localdomain/35207] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-12 04:02:25,158 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:02:25,160 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-12 04:02:25,161 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testInvalidSplitKey 2023-02-12 04:02:25,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:25,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-12 04:02:25,164 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174545164"}]},"ts":"1676174545164"} 2023-02-12 04:02:25,165 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-12 04:02:25,166 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-12 04:02:25,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=35ea055f2d9a2a9b0e631baf23e56ecf, UNASSIGN}] 2023-02-12 04:02:25,168 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=35ea055f2d9a2a9b0e631baf23e56ecf, UNASSIGN 2023-02-12 04:02:25,169 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=35ea055f2d9a2a9b0e631baf23e56ecf, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:25,169 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676174545169"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174545169"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174545169"}]},"ts":"1676174545169"} 2023-02-12 04:02:25,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 35ea055f2d9a2a9b0e631baf23e56ecf, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:02:25,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-12 04:02:25,323 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:25,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 35ea055f2d9a2a9b0e631baf23e56ecf, disabling compactions & flushes 2023-02-12 04:02:25,324 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:25,324 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:25,324 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. after waiting 0 ms 2023-02-12 04:02:25,324 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:25,332 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-12 04:02:25,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:25,334 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf. 2023-02-12 04:02:25,334 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 35ea055f2d9a2a9b0e631baf23e56ecf: Waiting for close lock at 1676174545323Running coprocessor pre-close hooks at 1676174545323Disabling compacts and flushes for region at 1676174545323Disabling writes for close at 1676174545324 (+1 ms)Writing region close event to WAL at 1676174545326 (+2 ms)Running coprocessor post-close hooks at 1676174545333 (+7 ms)Closed at 1676174545334 (+1 ms) 2023-02-12 04:02:25,336 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:25,336 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=35ea055f2d9a2a9b0e631baf23e56ecf, regionState=CLOSED 2023-02-12 04:02:25,336 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676174545336"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174545336"}]},"ts":"1676174545336"} 2023-02-12 04:02:25,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-12 04:02:25,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 35ea055f2d9a2a9b0e631baf23e56ecf, server=jenkins-hbase20.apache.org,43977,1676174449184 in 166 msec 2023-02-12 04:02:25,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-12 04:02:25,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=35ea055f2d9a2a9b0e631baf23e56ecf, UNASSIGN in 172 msec 2023-02-12 04:02:25,342 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174545342"}]},"ts":"1676174545342"} 2023-02-12 04:02:25,343 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-12 04:02:25,344 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-12 04:02:25,346 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 184 msec 2023-02-12 04:02:25,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-12 04:02:25,468 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-12 04:02:25,470 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testInvalidSplitKey 2023-02-12 04:02:25,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:25,475 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:25,476 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:25,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-12 04:02:25,481 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:25,483 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/recovered.edits] 2023-02-12 04:02:25,489 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/1e41abd4aa6642df82bd045d0dfa5ef3 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/1e41abd4aa6642df82bd045d0dfa5ef3 2023-02-12 04:02:25,491 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/3d6bbe08df704aa7a83678454b067d05 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/3d6bbe08df704aa7a83678454b067d05 2023-02-12 04:02:25,493 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/6944369283534277b8d50ed63799e024 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/6944369283534277b8d50ed63799e024 2023-02-12 04:02:25,495 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/b3ac813e0f1f469ea546b6ebea87cc03 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/b3ac813e0f1f469ea546b6ebea87cc03 2023-02-12 04:02:25,496 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/ce28a1662dc3498498e2a0a89ca465b5 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/ce28a1662dc3498498e2a0a89ca465b5 2023-02-12 04:02:25,498 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/e1d6ae305cd14079934505f506d79067 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/e1d6ae305cd14079934505f506d79067 2023-02-12 04:02:25,500 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/e84c73798c58441c91d004195629c650 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf1/e84c73798c58441c91d004195629c650 2023-02-12 04:02:25,504 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/0336854937624d46bf9090cd9dbced81 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/0336854937624d46bf9090cd9dbced81 2023-02-12 04:02:25,505 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/22c09705f1b94702990803ec235e0a1e to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/22c09705f1b94702990803ec235e0a1e 2023-02-12 04:02:25,507 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/7bfc88b3c18c4044920fa846071ee349 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/7bfc88b3c18c4044920fa846071ee349 2023-02-12 04:02:25,509 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/81fe6bd0846e4c58a1ec3410fd033a24 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/81fe6bd0846e4c58a1ec3410fd033a24 2023-02-12 04:02:25,511 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/91d4192175ad4cd5bc0f0184f9e838f7 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/91d4192175ad4cd5bc0f0184f9e838f7 2023-02-12 04:02:25,512 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/e6ae8b09cef34e0eb239789b5b0ea1f3 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/e6ae8b09cef34e0eb239789b5b0ea1f3 2023-02-12 04:02:25,514 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/ece11c370d624237957f1278a5118710 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/cf2/ece11c370d624237957f1278a5118710 2023-02-12 04:02:25,517 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/recovered.edits/85.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf/recovered.edits/85.seqid 2023-02-12 04:02:25,518 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testInvalidSplitKey/35ea055f2d9a2a9b0e631baf23e56ecf 2023-02-12 04:02:25,518 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-12 04:02:25,521 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:25,523 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-12 04:02:25,528 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-12 04:02:25,530 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:25,530 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-12 04:02:25,530 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174545530"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:25,532 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-12 04:02:25,532 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 35ea055f2d9a2a9b0e631baf23e56ecf, NAME => 'testInvalidSplitKey,,1676174543497.35ea055f2d9a2a9b0e631baf23e56ecf.', STARTKEY => '', ENDKEY => ''}] 2023-02-12 04:02:25,532 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-12 04:02:25,532 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676174545532"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:25,534 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-12 04:02:25,536 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:25,537 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 66 msec 2023-02-12 04:02:25,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-12 04:02:25,579 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-12 04:02:25,601 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=364 (was 342) - Thread LEAK? -, OpenFileDescriptor=738 (was 649) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=132 (was 127) - SystemLoadAverage LEAK? -, ProcessCount=163 (was 163), AvailableMemoryMB=4893 (was 4894) 2023-02-12 04:02:25,614 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=364, OpenFileDescriptor=738, MaxFileDescriptor=60000, SystemLoadAverage=132, ProcessCount=163, AvailableMemoryMB=4892 2023-02-12 04:02:25,614 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:02:25,617 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-12 04:02:25,617 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-12 04:02:25,618 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:25,620 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 04:02:25,621 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 04:02:25,622 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:25,623 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe empty. 2023-02-12 04:02:25,623 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:25,623 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-12 04:02:25,635 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-12 04:02:25,636 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3f29f95122f21236c2548f648f2e87fe, NAME => 'testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp 2023-02-12 04:02:25,647 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:25,648 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 3f29f95122f21236c2548f648f2e87fe, disabling compactions & flushes 2023-02-12 04:02:25,648 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:25,648 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:25,648 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. after waiting 0 ms 2023-02-12 04:02:25,648 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:25,648 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:25,648 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 3f29f95122f21236c2548f648f2e87fe: Waiting for close lock at 1676174545648Disabling compacts and flushes for region at 1676174545648Disabling writes for close at 1676174545648Writing region close event to WAL at 1676174545648Closed at 1676174545648 2023-02-12 04:02:25,650 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 04:02:25,651 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174545651"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174545651"}]},"ts":"1676174545651"} 2023-02-12 04:02:25,652 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 04:02:25,652 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 04:02:25,653 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174545652"}]},"ts":"1676174545652"} 2023-02-12 04:02:25,653 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-12 04:02:25,655 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-12 04:02:25,656 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-12 04:02:25,656 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-12 04:02:25,656 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-12 04:02:25,656 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-12 04:02:25,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, ASSIGN}] 2023-02-12 04:02:25,658 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, ASSIGN 2023-02-12 04:02:25,658 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,37065,1676174449218; forceNewPlan=false, retain=false 2023-02-12 04:02:25,809 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-12 04:02:25,811 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=3f29f95122f21236c2548f648f2e87fe, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:25,811 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174545811"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174545811"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174545811"}]},"ts":"1676174545811"} 2023-02-12 04:02:25,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218}] 2023-02-12 04:02:25,895 ERROR [BP-664456210-148.251.75.209-1676174445046 heartbeating to localhost.localdomain/127.0.0.1:37867] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-664456210-148.251.75.209-1676174445046 (Datanode Uuid 6bee6fa0-f002-488e-898d-32eb09388f7e) service to localhost.localdomain/127.0.0.1:37867 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-12 04:02:25,971 DEBUG [RSProcedureDispatcher-pool-4] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:25,971 DEBUG [RSProcedureDispatcher-pool-4] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-12 04:02:25,978 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:51594, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-12 04:02:25,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:25,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3f29f95122f21236c2548f648f2e87fe, NAME => 'testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:02:25,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:02:25,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:25,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:25,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:25,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:25,988 INFO [StoreOpener-3f29f95122f21236c2548f648f2e87fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:25,990 DEBUG [StoreOpener-3f29f95122f21236c2548f648f2e87fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1 2023-02-12 04:02:25,990 DEBUG [StoreOpener-3f29f95122f21236c2548f648f2e87fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1 2023-02-12 04:02:25,990 INFO [StoreOpener-3f29f95122f21236c2548f648f2e87fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3f29f95122f21236c2548f648f2e87fe columnFamilyName cf1 2023-02-12 04:02:25,991 INFO [StoreOpener-3f29f95122f21236c2548f648f2e87fe-1] regionserver.HStore(310): Store=3f29f95122f21236c2548f648f2e87fe/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:25,991 INFO [StoreOpener-3f29f95122f21236c2548f648f2e87fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:25,992 DEBUG [StoreOpener-3f29f95122f21236c2548f648f2e87fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2 2023-02-12 04:02:25,992 DEBUG [StoreOpener-3f29f95122f21236c2548f648f2e87fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2 2023-02-12 04:02:25,992 INFO [StoreOpener-3f29f95122f21236c2548f648f2e87fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3f29f95122f21236c2548f648f2e87fe columnFamilyName cf2 2023-02-12 04:02:25,993 INFO [StoreOpener-3f29f95122f21236c2548f648f2e87fe-1] regionserver.HStore(310): Store=3f29f95122f21236c2548f648f2e87fe/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:25,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:25,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:25,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:02:25,997 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:25,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 04:02:25,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 3f29f95122f21236c2548f648f2e87fe; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63143666, jitterRate=-0.05908605456352234}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:02:25,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 3f29f95122f21236c2548f648f2e87fe: Running coprocessor pre-open hook at 1676174545987Writing region info on filesystem at 1676174545987Initializing all the Stores at 1676174545988 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174545988Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174545988Cleaning up temporary data from old regions at 1676174545995 (+7 ms)Cleaning up detritus from prior splits at 1676174545995Running coprocessor post-open hooks at 1676174545999 (+4 ms)Region opened successfully at 1676174545999 2023-02-12 04:02:26,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe., pid=111, masterSystemTime=1676174545971 2023-02-12 04:02:26,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:26,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:26,004 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=3f29f95122f21236c2548f648f2e87fe, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,005 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174546004"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174546004"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174546004"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174546004"}]},"ts":"1676174546004"} 2023-02-12 04:02:26,007 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-12 04:02:26,007 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 in 191 msec 2023-02-12 04:02:26,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-12 04:02:26,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, ASSIGN in 351 msec 2023-02-12 04:02:26,009 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 04:02:26,010 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174546009"}]},"ts":"1676174546009"} 2023-02-12 04:02:26,010 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-12 04:02:26,012 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 04:02:26,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 394 msec 2023-02-12 04:02:26,123 DEBUG [Listener at localhost.localdomain/35207] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 04:02:26,129 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:51606, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 04:02:26,134 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:26,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-12 04:02:26,135 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-12 04:02:26,135 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,135 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-12 04:02:26,135 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-12 04:02:26,136 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,136 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,136 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,136 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,136 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,136 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,137 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,137 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:26,138 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:26,138 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:26,138 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. started... 2023-02-12 04:02:26,140 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f29f95122f21236c2548f648f2e87fe 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-12 04:02:26,141 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,141 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,141 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,141 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,141 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,141 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,152 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/5967d4f87a46419a9ee899329e14c8a6 2023-02-12 04:02:26,167 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/6ef343dd02cc409596fceacac99e6939 2023-02-12 04:02:26,172 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/5967d4f87a46419a9ee899329e14c8a6 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6 2023-02-12 04:02:26,178 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:02:26,179 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/6ef343dd02cc409596fceacac99e6939 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939 2023-02-12 04:02:26,184 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:02:26,185 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3f29f95122f21236c2548f648f2e87fe in 45ms, sequenceid=6, compaction requested=false 2023-02-12 04:02:26,185 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-12 04:02:26,186 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f29f95122f21236c2548f648f2e87fe: 2023-02-12 04:02:26,186 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:26,186 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:26,186 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:26,186 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,186 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,186 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,188 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,188 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,188 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,188 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,191 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-12 04:02:26,191 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@492da98d[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:26,191 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-12 04:02:26,191 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,191 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,191 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,191 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,192 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,192 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:26,192 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,192 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,192 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:26,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,192 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,192 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,192 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,192 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:26,192 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,192 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:26,193 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,193 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,193 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,193 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,193 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,193 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,193 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,194 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,194 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,194 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,37065,1676174449218': 2023-02-12 04:02:26,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-12 04:02:26,197 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-12 04:02:26,197 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:26,197 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:26,197 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,197 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:26,198 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,198 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,198 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,198 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,198 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,198 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,198 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,198 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,198 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,199 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,200 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,200 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,200 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,202 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-12 04:02:26,202 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:26,202 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:26,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:26,202 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,203 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,203 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,203 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,203 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,203 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,303 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:26,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:26,323 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:26,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:26,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-12 04:02:26,325 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-12 04:02:26,325 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,325 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-12 04:02:26,325 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-12 04:02:26,325 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,326 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,326 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,326 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,326 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,326 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,327 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,327 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:26,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,328 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,328 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,328 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,328 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,328 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:26,328 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:26,328 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:26,328 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,328 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,328 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:26,329 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:26,329 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:26,329 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. started... 2023-02-12 04:02:26,329 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f29f95122f21236c2548f648f2e87fe 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:26,331 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,331 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,331 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,331 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,332 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,332 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,339 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/abf69c9a295d4de19d64233ce1e9ddd4 2023-02-12 04:02:26,357 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/068647bee9904ce592405390efa57dc6 2023-02-12 04:02:26,361 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/abf69c9a295d4de19d64233ce1e9ddd4 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4 2023-02-12 04:02:26,364 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:02:26,364 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/068647bee9904ce592405390efa57dc6 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6 2023-02-12 04:02:26,367 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:02:26,368 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3f29f95122f21236c2548f648f2e87fe in 39ms, sequenceid=19, compaction requested=false 2023-02-12 04:02:26,368 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f29f95122f21236c2548f648f2e87fe: 2023-02-12 04:02:26,368 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:26,368 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:26,368 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:26,368 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,368 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,368 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,369 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,369 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,369 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,369 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-12 04:02:26,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c7d1b04[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:26,371 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-12 04:02:26,371 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,372 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,372 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,372 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,372 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,372 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,372 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:26,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,372 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,372 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:26,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,372 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,372 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,372 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,372 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:26,372 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,372 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:26,373 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,373 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,373 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,373 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,373 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,373 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,373 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,374 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,374 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,374 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,37065,1676174449218': 2023-02-12 04:02:26,376 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-12 04:02:26,376 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-12 04:02:26,377 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:26,377 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:26,377 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,377 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:26,377 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,377 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,377 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,377 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,377 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,378 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,378 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,378 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,378 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,379 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,380 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,381 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,381 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,383 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,383 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,383 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,383 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,383 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,383 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,383 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,383 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,383 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-12 04:02:26,383 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:26,383 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,383 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,383 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,384 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:26,384 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:26,383 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,384 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,384 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,384 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,384 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,384 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,384 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,484 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:26,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:26,504 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:26,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:26,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-12 04:02:26,506 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-12 04:02:26,506 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,506 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-12 04:02:26,506 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-12 04:02:26,507 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,507 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,508 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,508 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,508 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,508 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,508 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,508 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:26,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:26,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,511 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,511 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,511 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,511 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:26,511 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,511 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:26,511 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:26,511 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:26,511 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:26,511 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,511 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,511 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,511 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,511 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,511 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,511 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. started... 2023-02-12 04:02:26,511 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,511 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,512 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,512 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f29f95122f21236c2548f648f2e87fe 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:26,511 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,512 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,513 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,513 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,513 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,513 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,513 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,513 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,523 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/a684eba7afe44eb8a6a30e24c8c5e206 2023-02-12 04:02:26,541 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/925b71ec9aad46cc9b2f702082dd4dfb 2023-02-12 04:02:26,549 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/a684eba7afe44eb8a6a30e24c8c5e206 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206 2023-02-12 04:02:26,555 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:02:26,556 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/925b71ec9aad46cc9b2f702082dd4dfb as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb 2023-02-12 04:02:26,562 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:02:26,564 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3f29f95122f21236c2548f648f2e87fe in 51ms, sequenceid=32, compaction requested=true 2023-02-12 04:02:26,564 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f29f95122f21236c2548f648f2e87fe: 2023-02-12 04:02:26,564 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:26,564 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:26,564 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:26,564 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,564 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,564 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,565 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,565 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,566 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,566 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,567 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-12 04:02:26,567 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c58ff21[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:26,567 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-12 04:02:26,568 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,568 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,568 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,568 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,568 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,568 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,568 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,568 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,569 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:26,568 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,568 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:26,568 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,569 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,569 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:26,569 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:26,569 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,569 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,569 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,570 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,570 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,570 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,570 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,570 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,570 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,571 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,37065,1676174449218': 2023-02-12 04:02:26,574 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-12 04:02:26,574 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-12 04:02:26,574 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:26,574 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:26,574 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,574 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:26,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,578 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,578 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,579 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,579 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,581 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,581 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,581 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,587 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,587 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,587 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,587 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,587 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-12 04:02:26,587 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:26,587 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,587 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,587 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,587 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,587 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,587 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,587 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:26,587 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,588 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:26,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,588 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,588 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,588 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,588 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,588 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,588 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,588 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,688 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:26,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:26,704 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:26,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:26,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-12 04:02:26,706 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-12 04:02:26,706 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,706 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-12 04:02:26,706 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-12 04:02:26,707 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,707 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,709 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,709 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,709 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,709 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,709 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,709 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:26,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:26,710 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,710 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:26,710 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,710 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,710 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,710 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,710 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,710 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,711 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,711 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,711 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,711 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:26,711 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:26,711 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,711 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:26,711 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,711 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,711 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. started... 2023-02-12 04:02:26,711 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f29f95122f21236c2548f648f2e87fe 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:26,711 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,712 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,712 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,712 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,712 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,712 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,738 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/4fe6f413e36149b7a69ec5024bc22db4 2023-02-12 04:02:26,748 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/5d12de4367bf4ca4adab753145aca601 2023-02-12 04:02:26,752 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/4fe6f413e36149b7a69ec5024bc22db4 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4 2023-02-12 04:02:26,755 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:02:26,756 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/5d12de4367bf4ca4adab753145aca601 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601 2023-02-12 04:02:26,759 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:02:26,760 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3f29f95122f21236c2548f648f2e87fe in 49ms, sequenceid=45, compaction requested=true 2023-02-12 04:02:26,760 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f29f95122f21236c2548f648f2e87fe: 2023-02-12 04:02:26,760 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:26,760 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:26,760 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:26,760 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,760 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,760 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,761 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,761 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,761 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,761 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,763 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-12 04:02:26,763 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4539fb57[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:26,763 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-12 04:02:26,763 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,764 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,764 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,764 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,764 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,764 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:26,764 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,764 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,764 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,764 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,764 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,764 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:26,764 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,764 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:26,764 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:26,765 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,765 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,765 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,765 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,765 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,765 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,765 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,765 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,765 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,765 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,37065,1676174449218': 2023-02-12 04:02:26,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-12 04:02:26,768 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-12 04:02:26,768 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:26,768 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:26,768 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,768 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:26,769 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,769 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,769 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,769 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,769 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,769 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,769 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,769 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,770 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,770 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,771 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,771 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,772 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,773 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,773 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,773 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,773 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,773 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,773 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,773 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-12 04:02:26,774 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:26,773 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,773 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,773 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,774 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:26,773 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,774 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:26,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,773 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,774 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,774 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,774 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,774 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,774 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,774 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,774 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,874 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:26,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:26,894 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:26,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:26,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-12 04:02:26,896 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-12 04:02:26,896 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,896 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-12 04:02:26,896 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-12 04:02:26,897 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,897 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,898 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,898 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,898 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,898 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,898 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,899 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:26,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,899 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,899 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,899 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,900 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:26,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,900 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,900 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:26,900 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:26,900 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,900 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:26,900 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,900 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:26,900 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,900 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,900 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,900 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,901 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,901 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,901 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,901 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,900 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,901 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,901 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:26,901 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:26,901 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,901 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:26,901 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:26,901 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:26,901 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:26,901 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. started... 2023-02-12 04:02:26,902 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f29f95122f21236c2548f648f2e87fe 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:26,903 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,903 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,903 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,903 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,903 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,903 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,911 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/691e7fe42cfe4a6aba77744f61453ba3 2023-02-12 04:02:26,926 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/1ae8801e9bbf4322893429dcbd0a2a9a 2023-02-12 04:02:26,932 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/691e7fe42cfe4a6aba77744f61453ba3 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3 2023-02-12 04:02:26,937 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:02:26,938 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/1ae8801e9bbf4322893429dcbd0a2a9a as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a 2023-02-12 04:02:26,942 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:02:26,943 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3f29f95122f21236c2548f648f2e87fe in 42ms, sequenceid=58, compaction requested=true 2023-02-12 04:02:26,943 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f29f95122f21236c2548f648f2e87fe: 2023-02-12 04:02:26,943 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:26,943 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:26,944 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:26,944 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,944 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:26,944 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:26,945 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,945 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,945 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,945 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:26,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,947 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-12 04:02:26,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61fc53bc[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:26,948 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-12 04:02:26,948 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,948 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,948 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,948 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,949 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,949 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,949 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,949 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:26,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,949 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:26,949 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,949 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,949 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:26,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,949 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:26,949 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:26,949 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:26,950 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,950 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,950 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,950 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,950 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,950 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,950 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,950 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:26,950 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,950 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:26,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,37065,1676174449218': 2023-02-12 04:02:26,954 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-12 04:02:26,954 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-12 04:02:26,954 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:26,954 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:26,954 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-12 04:02:26,954 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:26,955 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,955 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:26,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:26,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,955 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,955 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,955 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,955 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,955 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:26,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,956 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,956 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,957 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:26,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:26,958 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:26,958 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,959 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:26,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,961 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-12 04:02:26,961 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:26,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,961 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:26,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:26,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:26,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:26,962 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:26,961 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,962 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:26,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:26,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,962 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:26,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:26,962 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,963 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:26,963 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:26,963 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:26,963 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,062 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:27,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:27,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:27,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:27,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-12 04:02:27,086 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-12 04:02:27,086 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:27,086 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-12 04:02:27,086 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-12 04:02:27,086 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,086 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,087 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:27,087 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:27,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:27,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:27,087 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,087 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:27,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:27,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:27,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:27,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:27,088 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,088 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:27,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:27,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:27,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:27,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,089 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:27,088 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:27,089 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-12 04:02:27,089 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:27,089 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:27,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:27,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:27,089 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:27,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:27,089 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:27,089 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. started... 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:27,089 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:27,090 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3f29f95122f21236c2548f648f2e87fe 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:27,090 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:27,090 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,091 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,091 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,091 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:27,091 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,091 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:27,102 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/c7182a61c5314f7cad6a0180171bc553 2023-02-12 04:02:27,111 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/fc3698530644446f9a22f4915ea723e2 2023-02-12 04:02:27,115 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/c7182a61c5314f7cad6a0180171bc553 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553 2023-02-12 04:02:27,118 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:02:27,118 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/fc3698530644446f9a22f4915ea723e2 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2 2023-02-12 04:02:27,122 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:02:27,122 INFO [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3f29f95122f21236c2548f648f2e87fe in 32ms, sequenceid=71, compaction requested=true 2023-02-12 04:02:27,123 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3f29f95122f21236c2548f648f2e87fe: 2023-02-12 04:02:27,123 DEBUG [rs(jenkins-hbase20.apache.org,37065,1676174449218)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:27,123 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:27,123 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:27,123 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,123 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-12 04:02:27,123 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-12 04:02:27,124 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,124 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:27,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:27,125 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:27,125 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:27,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-12 04:02:27,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@251dd1bc[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:27,126 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-12 04:02:27,126 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,127 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,127 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,127 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,127 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:27,127 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:27,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,127 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:27,127 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,127 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,127 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:27,128 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:27,128 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-12 04:02:27,128 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-12 04:02:27,128 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:27,128 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:27,128 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:27,128 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:27,128 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:27,129 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:27,129 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-12 04:02:27,129 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:27,129 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,129 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:27,129 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:27,129 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-12 04:02:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,37065,1676174449218': 2023-02-12 04:02:27,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-12 04:02:27,132 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-12 04:02:27,132 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:27,132 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:27,132 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-12 04:02:27,132 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:27,136 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,136 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:27,136 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,137 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:27,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,136 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,136 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,137 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:27,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:27,137 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:27,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:27,137 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,138 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:27,139 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-12 04:02:27,140 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:27,140 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:27,145 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:27,145 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,145 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:27,145 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,145 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:27,145 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:27,145 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:27,146 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:27,145 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-12 04:02:27,146 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:27,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:27,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:27,145 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:27,145 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:27,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:27,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:27,145 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:27,146 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:27,146 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:27,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:27,146 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:27,146 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:27,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:27,147 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:27,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:27,147 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:27,147 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:27,147 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,147 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-12 04:02:27,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:27,247 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:27,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:27,516 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-12 04:02:27,519 DEBUG [Listener at localhost.localdomain/35207] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:27,521 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:27,523 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:27,524 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:27,524 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:27,525 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:02:27,526 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:27,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:02:27,526 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:02:27,771 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:27,772 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:02:27,772 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:02:27,772 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:02:27,773 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-12 04:02:27,773 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:02:27,774 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:02:27,774 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:02:27,774 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:02:27,776 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-12 04:02:27,780 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-12 04:02:27,781 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:02:27,782 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:02:27,783 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:02:27,783 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:02:27,783 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:27,783 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:02:27,783 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:27,783 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:27,784 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:27,784 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:02:27,784 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:27,784 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:27,784 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:27,784 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:02:27,784 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:02:27,785 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:27,785 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:27,785 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:27,785 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:02:27,785 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:27,785 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:27,786 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:27,786 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:02:27,786 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:27,786 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:27,786 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:27,786 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:27,787 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:27,787 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:27,787 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:27,787 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:27,787 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:27,787 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:27,788 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:27,788 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:27,788 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:02:27,788 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:27,788 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:27,788 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:27,788 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-12 04:02:27,789 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:02:27,790 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:02:27,790 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174577790 2023-02-12 04:02:27,791 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:27,791 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:02:27,793 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:02:27,794 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=3f29f95122f21236c2548f648f2e87fe, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37065,1676174449218, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218, openSeqNum=2 2023-02-12 04:02:27,794 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:02:27,794 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174607794 2023-02-12 04:02:27,794 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174667794 2023-02-12 04:02:27,794 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-12 04:02:27,794 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:02:27,796 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:02:27,796 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:27,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:02:27,796 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:02:28,044 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,045 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:02:28,045 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:02:28,045 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:02:28,046 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:02:28,046 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:02:28,046 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:02:28,046 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:02:28,047 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:02:28,048 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-12 04:02:28,053 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-12 04:02:28,053 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:02:28,054 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:02:28,055 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:02:28,055 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:02:28,055 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:28,055 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:02:28,055 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:28,055 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:28,055 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:28,056 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:02:28,056 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:28,056 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:28,056 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:28,056 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:02:28,056 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:02:28,056 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:28,056 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:28,057 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:28,057 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:02:28,057 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:28,057 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:28,057 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:28,057 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:02:28,057 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:28,057 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:28,057 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:28,058 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:28,058 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:28,058 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:28,058 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:28,058 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:28,058 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:28,058 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:28,058 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:28,059 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,059 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:02:28,059 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,059 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,059 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,059 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-12 04:02:28,060 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:02:28,060 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:02:28,060 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174578060 2023-02-12 04:02:28,062 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,062 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:02:28,064 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:02:28,064 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=3f29f95122f21236c2548f648f2e87fe, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37065,1676174449218, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218, openSeqNum=2 2023-02-12 04:02:28,064 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:02:28,064 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174608064 2023-02-12 04:02:28,064 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174668064 2023-02-12 04:02:28,064 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-12 04:02:28,064 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:02:28,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN}] 2023-02-12 04:02:28,066 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:02:28,066 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:02:28,066 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:02:28,314 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,315 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:02:28,315 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:02:28,315 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:02:28,315 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:02:28,315 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:02:28,315 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:02:28,315 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:02:28,316 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:02:28,317 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:02:28,319 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-12 04:02:28,320 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:02:28,320 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:02:28,321 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:02:28,321 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:02:28,321 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:28,322 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:02:28,322 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:28,323 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:28,323 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:28,323 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:02:28,324 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:28,324 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:28,325 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:28,325 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:02:28,325 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:02:28,326 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:28,326 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:28,327 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:28,327 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:02:28,327 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:28,327 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:28,327 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:28,327 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:02:28,327 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:28,327 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:28,327 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:28,328 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:28,328 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:28,328 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:28,328 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:28,328 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:28,328 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:28,328 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:28,329 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:28,329 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,329 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:02:28,329 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,329 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,329 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,329 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-12 04:02:28,330 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:02:28,330 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:02:28,331 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174578331 2023-02-12 04:02:28,331 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:02:28,332 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,334 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:02:28,334 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=3f29f95122f21236c2548f648f2e87fe, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37065,1676174449218, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218, openSeqNum=2 2023-02-12 04:02:28,334 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:02:28,334 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174608334 2023-02-12 04:02:28,335 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174668335 2023-02-12 04:02:28,335 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:02:28,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN}] 2023-02-12 04:02:28,335 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:02:28,337 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN 2023-02-12 04:02:28,337 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=3f29f95122f21236c2548f648f2e87fe, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:28,337 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174548337"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174548337"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174548337"}]},"ts":"1676174548337"} 2023-02-12 04:02:28,338 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218}] 2023-02-12 04:02:28,491 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 3f29f95122f21236c2548f648f2e87fe, disabling compactions & flushes 2023-02-12 04:02:28,492 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:28,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:28,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. after waiting 0 ms 2023-02-12 04:02:28,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:28,492 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 3f29f95122f21236c2548f648f2e87fe 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-12 04:02:28,510 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/8c84a00b00f44d2787bd95c3727be367 2023-02-12 04:02:28,530 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/132035159a0c4e82b45702c1160b54a7 2023-02-12 04:02:28,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf1/8c84a00b00f44d2787bd95c3727be367 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367 2023-02-12 04:02:28,536 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:02:28,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/.tmp/cf2/132035159a0c4e82b45702c1160b54a7 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7 2023-02-12 04:02:28,540 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:02:28,541 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3f29f95122f21236c2548f648f2e87fe in 49ms, sequenceid=82, compaction requested=true 2023-02-12 04:02:28,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-12 04:02:28,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:28,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. 2023-02-12 04:02:28,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 3f29f95122f21236c2548f648f2e87fe: Waiting for close lock at 1676174548492Running coprocessor pre-close hooks at 1676174548492Disabling compacts and flushes for region at 1676174548492Disabling writes for close at 1676174548492Obtaining lock to block concurrent updates at 1676174548493 (+1 ms)Preparing flush snapshotting stores in 3f29f95122f21236c2548f648f2e87fe at 1676174548493Finished memstore snapshotting testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676174548493Flushing stores of testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. at 1676174548494 (+1 ms)Flushing 3f29f95122f21236c2548f648f2e87fe/cf1: creating writer at 1676174548496 (+2 ms)Flushing 3f29f95122f21236c2548f648f2e87fe/cf1: appending metadata at 1676174548500 (+4 ms)Flushing 3f29f95122f21236c2548f648f2e87fe/cf1: closing flushed file at 1676174548500Flushing 3f29f95122f21236c2548f648f2e87fe/cf2: creating writer at 1676174548518 (+18 ms)Flushing 3f29f95122f21236c2548f648f2e87fe/cf2: appending metadata at 1676174548521 (+3 ms)Flushing 3f29f95122f21236c2548f648f2e87fe/cf2: closing flushed file at 1676174548521Flushing 3f29f95122f21236c2548f648f2e87fe/cf1: reopening flushed file at 1676174548534 (+13 ms)Flushing 3f29f95122f21236c2548f648f2e87fe/cf2: reopening flushed file at 1676174548537 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3f29f95122f21236c2548f648f2e87fe in 49ms, sequenceid=82, compaction requested=true at 1676174548541 (+4 ms)Writing region close event to WAL at 1676174548541Running coprocessor post-close hooks at 1676174548544 (+3 ms)Closed at 1676174548545 (+1 ms) 2023-02-12 04:02:28,546 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,547 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=3f29f95122f21236c2548f648f2e87fe, regionState=CLOSED 2023-02-12 04:02:28,547 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174548547"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174548547"}]},"ts":"1676174548547"} 2023-02-12 04:02:28,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-12 04:02:28,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 in 210 msec 2023-02-12 04:02:28,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-12 04:02:28,550 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN in 214 msec 2023-02-12 04:02:28,551 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:02:28,551 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:02:28,551 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:02:28,585 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,585 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:02:28,585 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:02:28,585 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:02:28,585 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:02:28,586 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:02:28,586 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:02:28,586 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:02:28,586 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:02:28,587 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:02:28,590 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-12 04:02:28,590 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:02:28,590 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:02:28,590 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:02:28,590 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:02:28,591 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:02:28,591 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:02:28,591 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:28,591 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:02:28,592 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:28,592 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:28,592 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:28,592 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:02:28,593 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:28,593 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:28,593 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:28,593 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:02:28,594 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:02:28,594 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:28,594 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:28,594 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:28,595 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:02:28,595 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:28,595 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:28,595 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:28,596 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:02:28,596 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:28,596 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:28,596 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:28,597 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:28,597 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:28,597 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:28,597 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:28,598 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:28,598 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:28,598 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:28,598 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:28,599 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,599 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN 2023-02-12 04:02:28,599 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:28,599 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:02:28,599 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,599 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,599 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,599 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:28,599 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:28,599 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-12 04:02:28,600 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:02:28,600 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:02:28,600 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174578600 2023-02-12 04:02:28,601 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:02:28,602 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:28,604 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:02:28,604 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=3f29f95122f21236c2548f648f2e87fe, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37065,1676174449218, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218, openSeqNum=2 2023-02-12 04:02:28,604 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:02:28,604 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174608604 2023-02-12 04:02:28,604 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174668604 2023-02-12 04:02:28,604 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:02:28,607 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:02:28,614 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=3f29f95122f21236c2548f648f2e87fe, threads=14 2023-02-12 04:02:28,615 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,615 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,615 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,615 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,615 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,615 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,615 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,615 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,616 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,616 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,616 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,616 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,616 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,616 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,745 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 04:02:28,751 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,762 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,783 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,800 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,801 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,814 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,815 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,815 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,822 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,831 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,831 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,831 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,835 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:28,894 ERROR [BP-664456210-148.251.75.209-1676174445046 heartbeating to localhost.localdomain/127.0.0.1:37867] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-664456210-148.251.75.209-1676174445046 (Datanode Uuid 484548f8-2ed9-47b7-81ed-a4821ba20939) service to localhost.localdomain/127.0.0.1:37867 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-12 04:02:29,210 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,210 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 3f29f95122f21236c2548f648f2e87fe Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-12 04:02:29,264 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:02:29,264 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:29,264 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:02:29,264 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:02:29,355 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:29,355 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:02:29,355 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:02:29,356 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:02:29,356 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:02:29,356 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:02:29,356 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:02:29,357 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:02:29,357 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:02:29,358 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-12 04:02:29,361 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-12 04:02:29,362 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:02:29,362 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:02:29,362 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:02:29,363 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:02:29,363 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:02:29,364 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:02:29,364 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:29,365 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:02:29,365 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:29,366 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:29,366 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:29,367 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:02:29,367 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:29,367 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:29,367 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:29,367 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:02:29,367 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:02:29,367 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:29,367 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:29,368 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:29,368 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:02:29,368 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:29,368 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:29,368 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:29,368 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:02:29,368 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:29,368 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:29,368 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:29,369 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:29,369 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:29,369 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:29,369 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:29,369 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:29,369 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:29,369 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:29,369 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:29,370 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:29,370 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN 2023-02-12 04:02:29,370 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:29,370 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:02:29,370 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:29,370 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:29,370 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:29,370 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:29,370 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:29,370 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-12 04:02:29,371 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:02:29,371 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:02:29,372 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174579372 2023-02-12 04:02:29,373 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:29,373 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:02:29,375 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:02:29,375 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=3f29f95122f21236c2548f648f2e87fe, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37065,1676174449218, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218, openSeqNum=2 2023-02-12 04:02:29,375 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:02:29,375 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174609375 2023-02-12 04:02:29,375 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174669375 2023-02-12 04:02:29,375 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-12 04:02:29,383 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=3f29f95122f21236c2548f648f2e87fe, threads=14 2023-02-12 04:02:29,386 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,386 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,386 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,386 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,389 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,389 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,390 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,391 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,391 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,391 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,391 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,391 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,392 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,392 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,441 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,446 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,452 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,453 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,454 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,456 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,460 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,460 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,461 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,461 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,463 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,463 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,861 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,862 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6 for region: 3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:29,862 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 3f29f95122f21236c2548f648f2e87fe Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-12 04:02:29,928 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:02:29,934 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-12 04:02:29,937 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-12 04:02:29,937 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:02:29,937 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:29,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:02:29,937 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:02:30,125 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,126 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:02:30,126 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:02:30,126 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:02:30,127 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:02:30,127 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:02:30,127 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:02:30,127 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:02:30,127 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:02:30,129 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-12 04:02:30,132 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-12 04:02:30,132 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:02:30,133 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:02:30,133 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:02:30,134 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:02:30,134 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:02:30,135 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:02:30,135 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:30,136 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:02:30,136 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:30,137 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:30,137 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:30,138 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:02:30,138 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:30,138 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:30,138 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:30,139 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:02:30,139 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:02:30,139 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:30,139 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:30,139 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:30,139 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:02:30,139 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:30,139 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:30,140 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:30,140 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:02:30,140 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:30,140 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:30,140 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,140 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,140 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:30,140 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,140 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,141 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:30,141 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:30,141 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:30,141 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:30,141 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,141 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN 2023-02-12 04:02:30,142 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:30,142 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:02:30,142 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,142 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,142 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,142 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:30,142 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:30,142 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 13 msec 2023-02-12 04:02:30,143 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:02:30,144 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:02:30,144 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174580144 2023-02-12 04:02:30,145 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,145 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:02:30,148 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:02:30,149 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=3f29f95122f21236c2548f648f2e87fe, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37065,1676174449218, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218, openSeqNum=2 2023-02-12 04:02:30,149 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:02:30,149 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174610149 2023-02-12 04:02:30,149 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174670149 2023-02-12 04:02:30,149 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:02:30,151 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:02:30,153 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:02:30,153 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:02:30,153 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:02:30,399 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,399 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:02:30,399 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:02:30,400 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:02:30,400 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:02:30,400 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:02:30,400 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:02:30,400 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:02:30,401 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:02:30,402 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-12 04:02:30,404 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-12 04:02:30,405 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:02:30,405 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:02:30,405 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:02:30,406 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:02:30,406 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:02:30,406 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:02:30,407 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:30,407 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:02:30,407 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:30,407 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:30,408 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:30,408 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:02:30,408 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:30,408 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:30,409 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:30,409 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:02:30,409 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:02:30,409 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:30,410 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:30,410 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:30,410 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:02:30,410 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:30,410 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:30,411 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:30,411 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:02:30,411 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:30,411 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:30,411 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,411 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,412 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:30,412 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,412 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,412 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:30,412 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:30,412 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:30,413 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:30,413 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,413 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN 2023-02-12 04:02:30,413 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:30,413 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:02:30,413 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,413 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,413 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,414 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:30,414 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:30,414 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-12 04:02:30,415 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:02:30,415 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:02:30,415 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174580415 2023-02-12 04:02:30,417 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,417 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:02:30,419 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:02:30,419 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=3f29f95122f21236c2548f648f2e87fe, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37065,1676174449218, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218, openSeqNum=2 2023-02-12 04:02:30,419 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:02:30,419 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174610419 2023-02-12 04:02:30,419 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174670419 2023-02-12 04:02:30,419 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-12 04:02:30,419 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:02:30,421 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174550421"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676174550421"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676174550421"}]},"ts":"1676174550421"} 2023-02-12 04:02:30,421 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676174550421"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174550421"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174550421"}]},"ts":"1676174550421"} 2023-02-12 04:02:30,421 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676174550421"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174550421"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174550421"}]},"ts":"1676174550421"} 2023-02-12 04:02:30,422 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:02:30,423 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:02:30,423 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:02:30,669 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,670 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:02:30,670 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:02:30,670 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:02:30,670 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:02:30,671 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:02:30,671 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:02:30,671 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:02:30,671 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:02:30,673 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-12 04:02:30,678 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-12 04:02:30,678 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:02:30,679 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:02:30,679 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:02:30,679 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:02:30,679 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:02:30,679 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:02:30,680 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:30,680 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:02:30,680 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:30,680 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:30,681 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:30,681 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:02:30,681 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:30,681 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:30,681 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:30,682 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:02:30,682 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:02:30,682 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:30,683 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:30,683 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:30,683 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:02:30,683 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:30,683 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:30,684 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:30,684 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:02:30,684 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:30,684 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:30,684 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,685 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,685 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:30,685 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,686 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,686 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:30,686 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:30,686 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:30,686 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:30,687 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,687 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN 2023-02-12 04:02:30,687 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:30,687 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:02:30,687 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,687 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,687 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,688 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:30,688 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:30,688 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 14 msec 2023-02-12 04:02:30,688 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:02:30,689 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:02:30,690 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174580690 2023-02-12 04:02:30,692 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,692 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:02:30,694 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:02:30,694 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=3f29f95122f21236c2548f648f2e87fe, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37065,1676174449218, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218, openSeqNum=2 2023-02-12 04:02:30,694 INFO [Listener at localhost.localdomain/35207] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,37065,1676174449218, table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:30,694 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=1c2f9bd3d076d7eaf00a9edbce758309, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-12 04:02:30,695 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=10d5bf8f68e9b321f926505e763d4c3c, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-12 04:02:30,695 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:02:30,695 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174610695 2023-02-12 04:02:30,695 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174670695 2023-02-12 04:02:30,695 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:02:30,695 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676174550695"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676174550695"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676174550695"}]},"ts":"1676174550695"} 2023-02-12 04:02:30,695 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676174550695"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174550695"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174550695"}]},"ts":"1676174550695"} 2023-02-12 04:02:30,695 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676174550695"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174550695"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174550695"}]},"ts":"1676174550695"} 2023-02-12 04:02:30,697 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:02:30,699 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:02:30,699 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:02:30,699 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:02:30,945 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,945 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:02:30,945 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:02:30,945 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:02:30,946 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:02:30,946 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:02:30,946 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:02:30,946 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:02:30,946 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:02:30,947 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:02:30,949 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-12 04:02:30,949 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:02:30,949 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:02:30,949 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:02:30,950 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:02:30,950 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:02:30,950 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:02:30,951 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:30,951 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:02:30,951 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:30,951 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:30,952 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:30,952 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:02:30,952 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:30,952 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:30,953 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:30,953 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:02:30,953 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:02:30,953 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:30,953 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:30,954 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:30,954 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:02:30,954 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:30,954 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:30,955 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:30,955 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:02:30,955 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:30,955 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:30,955 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,956 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,956 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:30,956 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,956 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:30,957 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:30,957 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:30,957 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:30,957 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:30,958 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,958 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN 2023-02-12 04:02:30,958 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:30,958 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:02:30,958 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,958 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,959 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,959 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:30,959 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:30,959 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-12 04:02:30,960 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:02:30,960 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:02:30,961 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174580961 2023-02-12 04:02:30,963 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:02:30,963 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,965 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:02:30,965 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=3f29f95122f21236c2548f648f2e87fe, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37065,1676174449218, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218, openSeqNum=2 2023-02-12 04:02:30,965 INFO [Listener at localhost.localdomain/35207] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,37065,1676174449218, table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:30,965 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=1c2f9bd3d076d7eaf00a9edbce758309, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-12 04:02:30,965 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=10d5bf8f68e9b321f926505e763d4c3c, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-12 04:02:30,965 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:02:30,965 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174610965 2023-02-12 04:02:30,965 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174670965 2023-02-12 04:02:30,965 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-12 04:02:30,965 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:02:30,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1c2f9bd3d076d7eaf00a9edbce758309, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=10d5bf8f68e9b321f926505e763d4c3c, ASSIGN}] 2023-02-12 04:02:30,967 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:02:30,967 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:30,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:02:30,967 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:02:31,216 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,216 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:02:31,216 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:02:31,216 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:02:31,217 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:02:31,217 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:02:31,217 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:02:31,218 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:02:31,218 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:02:31,220 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-12 04:02:31,223 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-12 04:02:31,223 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:02:31,223 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:02:31,223 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:02:31,223 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:02:31,224 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:02:31,224 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:02:31,224 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:31,224 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:02:31,224 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:31,224 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:31,225 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:31,225 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:02:31,225 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:31,225 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:31,225 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:31,225 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:02:31,226 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:02:31,226 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:31,226 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:31,226 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:31,226 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:02:31,226 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:31,227 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:31,227 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:31,227 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:02:31,227 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:31,227 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:31,227 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:31,228 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:31,228 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:31,228 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:31,228 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:31,229 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:31,229 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:31,229 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:31,229 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:31,229 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,229 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN 2023-02-12 04:02:31,229 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:31,230 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:02:31,230 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,230 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,230 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,230 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:31,230 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:31,230 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-12 04:02:31,231 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:02:31,232 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:02:31,232 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174581232 2023-02-12 04:02:31,234 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:02:31,234 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,236 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:02:31,236 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=3f29f95122f21236c2548f648f2e87fe, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37065,1676174449218, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218, openSeqNum=2 2023-02-12 04:02:31,236 INFO [Listener at localhost.localdomain/35207] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,37065,1676174449218, table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:31,236 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=1c2f9bd3d076d7eaf00a9edbce758309, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-12 04:02:31,237 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=10d5bf8f68e9b321f926505e763d4c3c, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-12 04:02:31,237 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:02:31,237 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174611237 2023-02-12 04:02:31,237 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174671237 2023-02-12 04:02:31,237 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:02:31,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1c2f9bd3d076d7eaf00a9edbce758309, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=10d5bf8f68e9b321f926505e763d4c3c, ASSIGN}] 2023-02-12 04:02:31,237 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:02:31,239 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=10d5bf8f68e9b321f926505e763d4c3c, ASSIGN 2023-02-12 04:02:31,239 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=10d5bf8f68e9b321f926505e763d4c3c, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,37065,1676174449218; forceNewPlan=false, retain=false 2023-02-12 04:02:31,240 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1c2f9bd3d076d7eaf00a9edbce758309, ASSIGN 2023-02-12 04:02:31,240 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1c2f9bd3d076d7eaf00a9edbce758309, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,37065,1676174449218; forceNewPlan=false, retain=false 2023-02-12 04:02:31,390 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-12 04:02:31,392 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=10d5bf8f68e9b321f926505e763d4c3c, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:31,393 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676174551392"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174551392"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174551392"}]},"ts":"1676174551392"} 2023-02-12 04:02:31,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure 10d5bf8f68e9b321f926505e763d4c3c, server=jenkins-hbase20.apache.org,37065,1676174449218}] 2023-02-12 04:02:31,401 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=1c2f9bd3d076d7eaf00a9edbce758309, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:31,401 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676174551401"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174551401"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174551401"}]},"ts":"1676174551401"} 2023-02-12 04:02:31,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure 1c2f9bd3d076d7eaf00a9edbce758309, server=jenkins-hbase20.apache.org,37065,1676174449218}] 2023-02-12 04:02:31,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c. 2023-02-12 04:02:31,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 10d5bf8f68e9b321f926505e763d4c3c, NAME => 'testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.', STARTKEY => '41', ENDKEY => ''} 2023-02-12 04:02:31,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:02:31,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:31,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,560 INFO [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,561 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1 2023-02-12 04:02:31,561 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1 2023-02-12 04:02:31,562 INFO [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 10d5bf8f68e9b321f926505e763d4c3c columnFamilyName cf1 2023-02-12 04:02:31,576 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4-top 2023-02-12 04:02:31,580 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6-top 2023-02-12 04:02:31,582 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3-top 2023-02-12 04:02:31,586 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367-top 2023-02-12 04:02:31,590 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206-top 2023-02-12 04:02:31,594 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4-top 2023-02-12 04:02:31,598 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553-top 2023-02-12 04:02:31,598 INFO [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(310): Store=10d5bf8f68e9b321f926505e763d4c3c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:31,598 INFO [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,599 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2 2023-02-12 04:02:31,599 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2 2023-02-12 04:02:31,599 INFO [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 10d5bf8f68e9b321f926505e763d4c3c columnFamilyName cf2 2023-02-12 04:02:31,617 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6-top 2023-02-12 04:02:31,621 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7-top 2023-02-12 04:02:31,625 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a-top 2023-02-12 04:02:31,628 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601-top 2023-02-12 04:02:31,632 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939-top 2023-02-12 04:02:31,635 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb-top 2023-02-12 04:02:31,638 DEBUG [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2-top 2023-02-12 04:02:31,639 INFO [StoreOpener-10d5bf8f68e9b321f926505e763d4c3c-1] regionserver.HStore(310): Store=10d5bf8f68e9b321f926505e763d4c3c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:31,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:02:31,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 10d5bf8f68e9b321f926505e763d4c3c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63079761, jitterRate=-0.06003831326961517}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:02:31,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 10d5bf8f68e9b321f926505e763d4c3c: Running coprocessor pre-open hook at 1676174551559Writing region info on filesystem at 1676174551559Initializing all the Stores at 1676174551560 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174551560Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174551560Cleaning up temporary data from old regions at 1676174551641 (+81 ms)Cleaning up detritus from prior splits at 1676174551641Running coprocessor post-open hooks at 1676174551643 (+2 ms)Region opened successfully at 1676174551643 2023-02-12 04:02:31,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c., pid=117, masterSystemTime=1676174551549 2023-02-12 04:02:31,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:31,647 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:02:31,647 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:02:31,647 INFO [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c. 2023-02-12 04:02:31,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:02:31,647 INFO [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c. 2023-02-12 04:02:31,647 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HStore(1912): 10d5bf8f68e9b321f926505e763d4c3c/cf2 is initiating minor compaction (all files) 2023-02-12 04:02:31,647 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HStore(1912): 10d5bf8f68e9b321f926505e763d4c3c/cf1 is initiating minor compaction (all files) 2023-02-12 04:02:31,647 INFO [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 10d5bf8f68e9b321f926505e763d4c3c/cf2 in testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c. 2023-02-12 04:02:31,647 INFO [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 10d5bf8f68e9b321f926505e763d4c3c/cf1 in testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c. 2023-02-12 04:02:31,648 INFO [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367-top] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/.tmp, totalSize=34.9 K 2023-02-12 04:02:31,648 INFO [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7-top] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/.tmp, totalSize=34.9 K 2023-02-12 04:02:31,648 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting 5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676174546130 2023-02-12 04:02:31,649 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting 6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676174546130 2023-02-12 04:02:31,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c. 2023-02-12 04:02:31,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c. 2023-02-12 04:02:31,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309. 2023-02-12 04:02:31,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1c2f9bd3d076d7eaf00a9edbce758309, NAME => 'testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.', STARTKEY => '', ENDKEY => '41'} 2023-02-12 04:02:31,649 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676174546306 2023-02-12 04:02:31,649 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=10d5bf8f68e9b321f926505e763d4c3c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:31,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:02:31,649 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676174546486 2023-02-12 04:02:31,649 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676174551649"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174551649"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174551649"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174551649"}]},"ts":"1676174551649"} 2023-02-12 04:02:31,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 1c2f9bd3d076d7eaf00a9edbce758309 2023-02-12 04:02:31,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:31,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 1c2f9bd3d076d7eaf00a9edbce758309 2023-02-12 04:02:31,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 1c2f9bd3d076d7eaf00a9edbce758309 2023-02-12 04:02:31,650 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting 4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676174546690 2023-02-12 04:02:31,650 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting 691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676174546876 2023-02-12 04:02:31,650 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting 068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676174546306 2023-02-12 04:02:31,651 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676174547065 2023-02-12 04:02:31,651 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting 925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676174546486 2023-02-12 04:02:31,651 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting 5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676174546690 2023-02-12 04:02:31,651 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting 8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676174547251 2023-02-12 04:02:31,652 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting 1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676174546876 2023-02-12 04:02:31,652 INFO [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1c2f9bd3d076d7eaf00a9edbce758309 2023-02-12 04:02:31,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-12 04:02:31,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 10d5bf8f68e9b321f926505e763d4c3c, server=jenkins-hbase20.apache.org,37065,1676174449218 in 255 msec 2023-02-12 04:02:31,653 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676174547065 2023-02-12 04:02:31,653 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting 132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676174547251 2023-02-12 04:02:31,654 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1 2023-02-12 04:02:31,655 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1 2023-02-12 04:02:31,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=10d5bf8f68e9b321f926505e763d4c3c, ASSIGN in 416 msec 2023-02-12 04:02:31,655 INFO [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1c2f9bd3d076d7eaf00a9edbce758309 columnFamilyName cf1 2023-02-12 04:02:31,664 INFO [RS:2;jenkins-hbase20:37065-shortCompactions-0] throttle.PressureAwareThroughputController(145): 10d5bf8f68e9b321f926505e763d4c3c#cf1#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:02:31,665 INFO [RS:2;jenkins-hbase20:37065-longCompactions-0] throttle.PressureAwareThroughputController(145): 10d5bf8f68e9b321f926505e763d4c3c#cf2#compaction#109 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:02:31,668 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4-bottom 2023-02-12 04:02:31,671 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6-bottom 2023-02-12 04:02:31,673 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3-bottom 2023-02-12 04:02:31,676 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367-bottom 2023-02-12 04:02:31,683 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206-bottom 2023-02-12 04:02:31,688 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4-bottom 2023-02-12 04:02:31,690 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553-bottom 2023-02-12 04:02:31,695 INFO [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(310): Store=1c2f9bd3d076d7eaf00a9edbce758309/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:31,695 INFO [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1c2f9bd3d076d7eaf00a9edbce758309 2023-02-12 04:02:31,696 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2 2023-02-12 04:02:31,696 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2 2023-02-12 04:02:31,696 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/.tmp/cf2/b73799f1ff944bc996fb51496a7304b5 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/b73799f1ff944bc996fb51496a7304b5 2023-02-12 04:02:31,696 INFO [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1c2f9bd3d076d7eaf00a9edbce758309 columnFamilyName cf2 2023-02-12 04:02:31,701 INFO [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 10d5bf8f68e9b321f926505e763d4c3c/cf2 of 10d5bf8f68e9b321f926505e763d4c3c into b73799f1ff944bc996fb51496a7304b5(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:02:31,701 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 10d5bf8f68e9b321f926505e763d4c3c: 2023-02-12 04:02:31,701 INFO [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c., storeName=10d5bf8f68e9b321f926505e763d4c3c/cf2, priority=9, startTime=1676174551647; duration=0sec 2023-02-12 04:02:31,701 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:31,704 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/.tmp/cf1/6abd6885b0f64174849e18763b405b98 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/6abd6885b0f64174849e18763b405b98 2023-02-12 04:02:31,708 INFO [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 10d5bf8f68e9b321f926505e763d4c3c/cf1 of 10d5bf8f68e9b321f926505e763d4c3c into 6abd6885b0f64174849e18763b405b98(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:02:31,708 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 10d5bf8f68e9b321f926505e763d4c3c: 2023-02-12 04:02:31,708 INFO [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c., storeName=10d5bf8f68e9b321f926505e763d4c3c/cf1, priority=9, startTime=1676174551644; duration=0sec 2023-02-12 04:02:31,708 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:31,709 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6-bottom 2023-02-12 04:02:31,712 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7-bottom 2023-02-12 04:02:31,714 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a-bottom 2023-02-12 04:02:31,717 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601-bottom 2023-02-12 04:02:31,720 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939-bottom 2023-02-12 04:02:31,722 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb-bottom 2023-02-12 04:02:31,725 DEBUG [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2-bottom 2023-02-12 04:02:31,725 INFO [StoreOpener-1c2f9bd3d076d7eaf00a9edbce758309-1] regionserver.HStore(310): Store=1c2f9bd3d076d7eaf00a9edbce758309/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:31,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309 2023-02-12 04:02:31,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309 2023-02-12 04:02:31,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:02:31,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 1c2f9bd3d076d7eaf00a9edbce758309 2023-02-12 04:02:31,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 1c2f9bd3d076d7eaf00a9edbce758309; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63225560, jitterRate=-0.05786573886871338}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:02:31,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 1c2f9bd3d076d7eaf00a9edbce758309: Running coprocessor pre-open hook at 1676174551650Writing region info on filesystem at 1676174551650Initializing all the Stores at 1676174551651 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174551651Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174551652 (+1 ms)Cleaning up temporary data from old regions at 1676174551727 (+75 ms)Cleaning up detritus from prior splits at 1676174551727Running coprocessor post-open hooks at 1676174551729 (+2 ms)Region opened successfully at 1676174551729 2023-02-12 04:02:31,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309., pid=118, masterSystemTime=1676174551549 2023-02-12 04:02:31,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:02:31,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:31,731 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:02:31,731 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:02:31,731 INFO [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309. 2023-02-12 04:02:31,731 INFO [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309. 2023-02-12 04:02:31,731 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HStore(1912): 1c2f9bd3d076d7eaf00a9edbce758309/cf1 is initiating minor compaction (all files) 2023-02-12 04:02:31,731 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HStore(1912): 1c2f9bd3d076d7eaf00a9edbce758309/cf2 is initiating minor compaction (all files) 2023-02-12 04:02:31,731 INFO [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1c2f9bd3d076d7eaf00a9edbce758309/cf1 in testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309. 2023-02-12 04:02:31,731 INFO [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1c2f9bd3d076d7eaf00a9edbce758309/cf2 in testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309. 2023-02-12 04:02:31,731 INFO [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367-bottom] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/.tmp, totalSize=34.9 K 2023-02-12 04:02:31,731 INFO [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7-bottom] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/.tmp, totalSize=34.9 K 2023-02-12 04:02:31,732 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting 5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174546130 2023-02-12 04:02:31,732 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting 6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174546130 2023-02-12 04:02:31,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309. 2023-02-12 04:02:31,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309. 2023-02-12 04:02:31,732 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174546306 2023-02-12 04:02:31,732 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting 068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174546306 2023-02-12 04:02:31,732 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174546486 2023-02-12 04:02:31,732 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=1c2f9bd3d076d7eaf00a9edbce758309, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:31,733 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676174551732"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174551732"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174551732"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174551732"}]},"ts":"1676174551732"} 2023-02-12 04:02:31,733 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting 925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174546486 2023-02-12 04:02:31,733 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting 4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174546690 2023-02-12 04:02:31,733 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting 5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174546690 2023-02-12 04:02:31,733 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting 691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174546876 2023-02-12 04:02:31,734 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting 1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174546876 2023-02-12 04:02:31,734 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174547065 2023-02-12 04:02:31,734 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174547065 2023-02-12 04:02:31,734 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] compactions.Compactor(207): Compacting 8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676174547251 2023-02-12 04:02:31,734 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] compactions.Compactor(207): Compacting 132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676174547251 2023-02-12 04:02:31,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-02-12 04:02:31,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 1c2f9bd3d076d7eaf00a9edbce758309, server=jenkins-hbase20.apache.org,37065,1676174449218 in 331 msec 2023-02-12 04:02:31,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-12 04:02:31,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1c2f9bd3d076d7eaf00a9edbce758309, ASSIGN in 499 msec 2023-02-12 04:02:31,737 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-12 04:02:31,737 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:02:31,737 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-12 04:02:31,737 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,737 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-12 04:02:31,737 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:02:31,737 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:02:31,738 ERROR [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-12 04:02:31,738 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-12 04:02:31,738 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-12 04:02:31,738 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-12 04:02:31,738 INFO [Listener at localhost.localdomain/35207] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 04:02:31,739 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 04:02:31,740 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-12 04:02:31,740 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-12 04:02:31,741 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-12 04:02:31,741 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 23] 2023-02-12 04:02:31,741 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-12 04:02:31,741 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 21] 2023-02-12 04:02:31,741 DEBUG [Listener at localhost.localdomain/35207] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-12 04:02:31,741 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-12 04:02:31,741 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-12 04:02:31,742 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-12 04:02:31,742 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-12 04:02:31,742 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:31,742 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=4157975c3727721623be5c93597b743a, daughterA=82e025ba906c703dbab0aa7ffe1c8599, daughterB=7cda275d8532d678cc77a56d10e4b57a 2023-02-12 04:02:31,742 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:31,742 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-12 04:02:31,743 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:31,743 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=14a5b26b7821b29d8e7405e41ed6325f, daughterA=2405ca844e0b3394a8e057a0063d11f6, daughterB=3d0c2ff996cdb076c77801003013a683 2023-02-12 04:02:31,743 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:31,743 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-12 04:02:31,743 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:31,743 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0dc25aecee603c971d1e12ebe66ec89b, daughterA=42a1d5b8ef83df4033d4e04086b0d8cc, daughterB=dc6259f2524b5fadfb0532a05454680b 2023-02-12 04:02:31,744 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0dc25aecee603c971d1e12ebe66ec89b, ASSIGN 2023-02-12 04:02:31,744 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:31,744 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-12 04:02:31,744 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:31,744 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ea775fa04e6f5b6170f1eb87d6c0efbc, daughterA=d6d457f69a40d1d02a5962902562a788, daughterB=6ced688347822f331e4682308ef38f26 2023-02-12 04:02:31,744 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:31,745 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-12 04:02:31,745 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:31,745 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=95c25e17def1a4a4a028fa2b6f6b5c57, daughterA=d5a95b4a9e2a7dc12ff3a61b5a4b7ab1, daughterB=e342c5ec8229f3d46866d60ac95bc270 2023-02-12 04:02:31,745 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:31,745 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-12 04:02:31,745 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:31,746 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:31,746 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43977,1676174449184, table=testRollbackForSplitTableRegionWithReplica, region=21357faa3e6d4ee08ffefe045d425842 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=612e24e4f6a2dc8f5218f7fab3e82c42, daughterA=c9baf4e9f9db6ea9302d5376f94531e4, daughterB=763b2faec8401ba8af9fd0aa931a32b7 2023-02-12 04:02:31,746 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:31,746 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-12 04:02:31,746 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:31,746 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:31,747 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-12 04:02:31,747 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:31,748 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,748 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN 2023-02-12 04:02:31,748 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:31,748 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1c2f9bd3d076d7eaf00a9edbce758309, ASSIGN 2023-02-12 04:02:31,748 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=10d5bf8f68e9b321f926505e763d4c3c, ASSIGN 2023-02-12 04:02:31,748 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 10d5bf8f68e9b321f926505e763d4c3c, server=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:31,748 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 1c2f9bd3d076d7eaf00a9edbce758309, server=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:31,749 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c held the lock before restarting, call acquireLock to restore it. 2023-02-12 04:02:31,749 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,749 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,749 INFO [Listener at localhost.localdomain/35207] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,749 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:31,749 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3f29f95122f21236c2548f648f2e87fe, server=jenkins-hbase20.apache.org,37065,1676174449218 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:31,749 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1c2f9bd3d076d7eaf00a9edbce758309, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:31,749 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=10d5bf8f68e9b321f926505e763d4c3c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:31,749 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 10d5bf8f68e9b321f926505e763d4c3c, server=jenkins-hbase20.apache.org,37065,1676174449218 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:31,749 DEBUG [Listener at localhost.localdomain/35207] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 1c2f9bd3d076d7eaf00a9edbce758309, server=jenkins-hbase20.apache.org,37065,1676174449218 didn't hold the lock before restarting, skip acquiring lock. 2023-02-12 04:02:31,749 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-12 04:02:31,750 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-12 04:02:31,750 INFO [RS:2;jenkins-hbase20:37065-longCompactions-0] throttle.PressureAwareThroughputController(145): 1c2f9bd3d076d7eaf00a9edbce758309#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:02:31,750 INFO [RS:2;jenkins-hbase20:37065-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1c2f9bd3d076d7eaf00a9edbce758309#cf1#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:02:31,750 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184, table=hbase:meta, region=1588230740 2023-02-12 04:02:31,754 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676174581754 2023-02-12 04:02:31,756 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:31,756 DEBUG [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 04:02:31,758 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=6b2660cb506f26cb2f51dda8314eff58, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43977,1676174449184, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184, openSeqNum=2 2023-02-12 04:02:31,758 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=3f29f95122f21236c2548f648f2e87fe, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,37065,1676174449218, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218, openSeqNum=2 2023-02-12 04:02:31,759 INFO [Listener at localhost.localdomain/35207] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,37065,1676174449218, table=testRecoveryAndDoubleExecution, region=3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:31,759 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=1c2f9bd3d076d7eaf00a9edbce758309, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37065,1676174449218, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218, openSeqNum=86 2023-02-12 04:02:31,759 DEBUG [Listener at localhost.localdomain/35207] assignment.RegionStateStore(147): Load hbase:meta entry region=10d5bf8f68e9b321f926505e763d4c3c, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,37065,1676174449218, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218, openSeqNum=86 2023-02-12 04:02:31,759 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-12 04:02:31,759 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676174611759 2023-02-12 04:02:31,759 INFO [Listener at localhost.localdomain/35207] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676174671759 2023-02-12 04:02:31,759 INFO [Listener at localhost.localdomain/35207] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-12 04:02:31,762 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-12 04:02:31,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3f29f95122f21236c2548f648f2e87fe, daughterA=1c2f9bd3d076d7eaf00a9edbce758309, daughterB=10d5bf8f68e9b321f926505e763d4c3c in 4.2430 sec 2023-02-12 04:02:31,776 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/.tmp/cf1/007d22bce9a2442e839a3b220f92968b as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/007d22bce9a2442e839a3b220f92968b 2023-02-12 04:02:31,776 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/.tmp/cf2/6e16015fd3bb47019ba7020c9e0cd082 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/6e16015fd3bb47019ba7020c9e0cd082 2023-02-12 04:02:31,780 INFO [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1c2f9bd3d076d7eaf00a9edbce758309/cf1 of 1c2f9bd3d076d7eaf00a9edbce758309 into 007d22bce9a2442e839a3b220f92968b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:02:31,780 INFO [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1c2f9bd3d076d7eaf00a9edbce758309/cf2 of 1c2f9bd3d076d7eaf00a9edbce758309 into 6e16015fd3bb47019ba7020c9e0cd082(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:02:31,780 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1c2f9bd3d076d7eaf00a9edbce758309: 2023-02-12 04:02:31,780 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1c2f9bd3d076d7eaf00a9edbce758309: 2023-02-12 04:02:31,780 INFO [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309., storeName=1c2f9bd3d076d7eaf00a9edbce758309/cf1, priority=9, startTime=1676174551730; duration=0sec 2023-02-12 04:02:31,780 INFO [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309., storeName=1c2f9bd3d076d7eaf00a9edbce758309/cf2, priority=9, startTime=1676174551730; duration=0sec 2023-02-12 04:02:31,780 DEBUG [RS:2;jenkins-hbase20:37065-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:31,780 DEBUG [RS:2;jenkins-hbase20:37065-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:32,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37065] ipc.CallRunner(144): callId: 858 service: ClientService methodName: Scan size: 154 connection: 148.251.75.209:51606 deadline: 1676174612013, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe. is not online on jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:32,127 DEBUG [Listener at localhost.localdomain/35207] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-12 04:02:32,133 INFO [Listener at localhost.localdomain/35207] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-12 04:02:32,134 DEBUG [Listener at localhost.localdomain/35207] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-12 04:02:32,144 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:02:32,145 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-12 04:02:32,146 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testRecoveryAndDoubleExecution 2023-02-12 04:02:32,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:32,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-12 04:02:32,149 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174552149"}]},"ts":"1676174552149"} 2023-02-12 04:02:32,149 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-12 04:02:32,150 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-12 04:02:32,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1c2f9bd3d076d7eaf00a9edbce758309, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=10d5bf8f68e9b321f926505e763d4c3c, UNASSIGN}] 2023-02-12 04:02:32,152 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=10d5bf8f68e9b321f926505e763d4c3c, UNASSIGN 2023-02-12 04:02:32,152 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=10d5bf8f68e9b321f926505e763d4c3c, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:32,152 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676174552152"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174552152"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174552152"}]},"ts":"1676174552152"} 2023-02-12 04:02:32,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 10d5bf8f68e9b321f926505e763d4c3c, server=jenkins-hbase20.apache.org,37065,1676174449218}] 2023-02-12 04:02:32,155 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1c2f9bd3d076d7eaf00a9edbce758309, UNASSIGN 2023-02-12 04:02:32,155 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=1c2f9bd3d076d7eaf00a9edbce758309, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:32,155 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676174552155"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174552155"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174552155"}]},"ts":"1676174552155"} 2023-02-12 04:02:32,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 1c2f9bd3d076d7eaf00a9edbce758309, server=jenkins-hbase20.apache.org,37065,1676174449218}] 2023-02-12 04:02:32,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-12 04:02:32,305 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:32,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 10d5bf8f68e9b321f926505e763d4c3c, disabling compactions & flushes 2023-02-12 04:02:32,305 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c. 2023-02-12 04:02:32,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c. 2023-02-12 04:02:32,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c. after waiting 0 ms 2023-02-12 04:02:32,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c. 2023-02-12 04:02:32,306 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367-top] to archive 2023-02-12 04:02:32,308 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:02:32,311 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,314 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,316 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,318 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,321 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,323 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,325 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,327 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7-top] to archive 2023-02-12 04:02:32,328 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:02:32,331 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,333 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,334 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,335 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,336 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,337 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,337 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-12 04:02:32,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:32,340 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c. 2023-02-12 04:02:32,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 10d5bf8f68e9b321f926505e763d4c3c: Waiting for close lock at 1676174552305Running coprocessor pre-close hooks at 1676174552305Disabling compacts and flushes for region at 1676174552305Disabling writes for close at 1676174552305Writing region close event to WAL at 1676174552338 (+33 ms)Running coprocessor post-close hooks at 1676174552340 (+2 ms)Closed at 1676174552340 2023-02-12 04:02:32,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:32,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 1c2f9bd3d076d7eaf00a9edbce758309 2023-02-12 04:02:32,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 1c2f9bd3d076d7eaf00a9edbce758309, disabling compactions & flushes 2023-02-12 04:02:32,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309. 2023-02-12 04:02:32,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309. 2023-02-12 04:02:32,342 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=10d5bf8f68e9b321f926505e763d4c3c, regionState=CLOSED 2023-02-12 04:02:32,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309. after waiting 0 ms 2023-02-12 04:02:32,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309. 2023-02-12 04:02:32,342 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676174552342"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174552342"}]},"ts":"1676174552342"} 2023-02-12 04:02:32,343 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367-bottom] to archive 2023-02-12 04:02:32,344 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:02:32,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-12 04:02:32,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 10d5bf8f68e9b321f926505e763d4c3c, server=jenkins-hbase20.apache.org,37065,1676174449218 in 191 msec 2023-02-12 04:02:32,346 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=10d5bf8f68e9b321f926505e763d4c3c, UNASSIGN in 194 msec 2023-02-12 04:02:32,346 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,347 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,348 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,349 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,351 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,352 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,353 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,355 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7-bottom] to archive 2023-02-12 04:02:32,356 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:02:32,357 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,359 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,360 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,361 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,363 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,364 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,366 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-12 04:02:32,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:32,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309. 2023-02-12 04:02:32,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 1c2f9bd3d076d7eaf00a9edbce758309: Waiting for close lock at 1676174552342Running coprocessor pre-close hooks at 1676174552342Disabling compacts and flushes for region at 1676174552342Disabling writes for close at 1676174552342Writing region close event to WAL at 1676174552367 (+25 ms)Running coprocessor post-close hooks at 1676174552371 (+4 ms)Closed at 1676174552371 2023-02-12 04:02:32,373 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 1c2f9bd3d076d7eaf00a9edbce758309 2023-02-12 04:02:32,374 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=1c2f9bd3d076d7eaf00a9edbce758309, regionState=CLOSED 2023-02-12 04:02:32,374 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676174552374"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174552374"}]},"ts":"1676174552374"} 2023-02-12 04:02:32,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-12 04:02:32,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 1c2f9bd3d076d7eaf00a9edbce758309, server=jenkins-hbase20.apache.org,37065,1676174449218 in 219 msec 2023-02-12 04:02:32,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-12 04:02:32,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1c2f9bd3d076d7eaf00a9edbce758309, UNASSIGN in 227 msec 2023-02-12 04:02:32,380 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174552380"}]},"ts":"1676174552380"} 2023-02-12 04:02:32,382 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-12 04:02:32,383 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-12 04:02:32,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 238 msec 2023-02-12 04:02:32,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-12 04:02:32,452 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-12 04:02:32,453 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testRecoveryAndDoubleExecution 2023-02-12 04:02:32,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:32,457 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:32,458 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:32,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-12 04:02:32,466 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,466 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309 2023-02-12 04:02:32,467 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:32,469 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/recovered.edits] 2023-02-12 04:02:32,470 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/recovered.edits] 2023-02-12 04:02:32,470 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/recovered.edits] 2023-02-12 04:02:32,474 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/.regioninfo to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/.regioninfo 2023-02-12 04:02:32,474 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/.regioninfo to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/.regioninfo 2023-02-12 04:02:32,475 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/4fe6f413e36149b7a69ec5024bc22db4 2023-02-12 04:02:32,476 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/5967d4f87a46419a9ee899329e14c8a6 2023-02-12 04:02:32,478 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,478 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/4fe6f413e36149b7a69ec5024bc22db4.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,478 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/691e7fe42cfe4a6aba77744f61453ba3 2023-02-12 04:02:32,479 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,479 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/5967d4f87a46419a9ee899329e14c8a6.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,479 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/8c84a00b00f44d2787bd95c3727be367 2023-02-12 04:02:32,480 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,480 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/691e7fe42cfe4a6aba77744f61453ba3.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,481 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/a684eba7afe44eb8a6a30e24c8c5e206 2023-02-12 04:02:32,481 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,482 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/8c84a00b00f44d2787bd95c3727be367.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,482 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/abf69c9a295d4de19d64233ce1e9ddd4 2023-02-12 04:02:32,482 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,483 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/a684eba7afe44eb8a6a30e24c8c5e206.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,483 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf1/c7182a61c5314f7cad6a0180171bc553 2023-02-12 04:02:32,483 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,484 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/abf69c9a295d4de19d64233ce1e9ddd4.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,485 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,491 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf1/c7182a61c5314f7cad6a0180171bc553.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,491 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/068647bee9904ce592405390efa57dc6 2023-02-12 04:02:32,492 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/132035159a0c4e82b45702c1160b54a7 2023-02-12 04:02:32,493 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,493 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/1ae8801e9bbf4322893429dcbd0a2a9a 2023-02-12 04:02:32,494 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/068647bee9904ce592405390efa57dc6.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,494 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,495 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/5d12de4367bf4ca4adab753145aca601 2023-02-12 04:02:32,495 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,496 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/132035159a0c4e82b45702c1160b54a7.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,496 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/6ef343dd02cc409596fceacac99e6939 2023-02-12 04:02:32,497 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,497 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/1ae8801e9bbf4322893429dcbd0a2a9a.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,498 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,498 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/925b71ec9aad46cc9b2f702082dd4dfb 2023-02-12 04:02:32,498 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/5d12de4367bf4ca4adab753145aca601.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,499 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/cf2/fc3698530644446f9a22f4915ea723e2 2023-02-12 04:02:32,499 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,499 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/6ef343dd02cc409596fceacac99e6939.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,501 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/1c2f9bd3d076d7eaf00a9edbce758309/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,501 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/925b71ec9aad46cc9b2f702082dd4dfb.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,502 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/recovered.edits/85.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe/recovered.edits/85.seqid 2023-02-12 04:02:32,502 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/10d5bf8f68e9b321f926505e763d4c3c/cf2/fc3698530644446f9a22f4915ea723e2.3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,502 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/3f29f95122f21236c2548f648f2e87fe 2023-02-12 04:02:32,503 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/007d22bce9a2442e839a3b220f92968b to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf1/007d22bce9a2442e839a3b220f92968b 2023-02-12 04:02:32,504 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/6abd6885b0f64174849e18763b405b98 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf1/6abd6885b0f64174849e18763b405b98 2023-02-12 04:02:32,505 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/6e16015fd3bb47019ba7020c9e0cd082 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/cf2/6e16015fd3bb47019ba7020c9e0cd082 2023-02-12 04:02:32,505 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/b73799f1ff944bc996fb51496a7304b5 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/cf2/b73799f1ff944bc996fb51496a7304b5 2023-02-12 04:02:32,507 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/recovered.edits/90.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309/recovered.edits/90.seqid 2023-02-12 04:02:32,507 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/1c2f9bd3d076d7eaf00a9edbce758309 2023-02-12 04:02:32,508 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/recovered.edits/90.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c/recovered.edits/90.seqid 2023-02-12 04:02:32,508 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testRecoveryAndDoubleExecution/10d5bf8f68e9b321f926505e763d4c3c 2023-02-12 04:02:32,508 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-12 04:02:32,510 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:32,512 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-12 04:02:32,513 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-12 04:02:32,514 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:32,514 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-12 04:02:32,514 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174552514"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:32,515 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174552514"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:32,515 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174552514"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:32,516 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-12 04:02:32,516 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3f29f95122f21236c2548f648f2e87fe, NAME => 'testRecoveryAndDoubleExecution,,1676174545618.3f29f95122f21236c2548f648f2e87fe.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 1c2f9bd3d076d7eaf00a9edbce758309, NAME => 'testRecoveryAndDoubleExecution,,1676174547519.1c2f9bd3d076d7eaf00a9edbce758309.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 10d5bf8f68e9b321f926505e763d4c3c, NAME => 'testRecoveryAndDoubleExecution,41,1676174547519.10d5bf8f68e9b321f926505e763d4c3c.', STARTKEY => '41', ENDKEY => ''}] 2023-02-12 04:02:32,516 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-12 04:02:32,516 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676174552516"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:32,517 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-12 04:02:32,519 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-12 04:02:32,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 65 msec 2023-02-12 04:02:32,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-12 04:02:32,561 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-12 04:02:32,582 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=376 (was 364) - Thread LEAK? -, OpenFileDescriptor=823 (was 738) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=122 (was 132), ProcessCount=163 (was 163), AvailableMemoryMB=4858 (was 4892) 2023-02-12 04:02:32,594 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376, OpenFileDescriptor=823, MaxFileDescriptor=60000, SystemLoadAverage=122, ProcessCount=163, AvailableMemoryMB=4857 2023-02-12 04:02:32,594 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:02:32,597 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-12 04:02:32,597 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-12 04:02:32,598 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-12 04:02:32,600 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 04:02:32,601 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 04:02:32,602 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:32,603 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4 empty. 2023-02-12 04:02:32,603 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:32,603 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-12 04:02:32,616 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-12 04:02:32,618 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 5a959618b9e9289355531d0e0306b9a4, NAME => 'testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp 2023-02-12 04:02:32,630 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:32,630 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 5a959618b9e9289355531d0e0306b9a4, disabling compactions & flushes 2023-02-12 04:02:32,631 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:32,631 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:32,631 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. after waiting 0 ms 2023-02-12 04:02:32,631 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:32,631 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:32,631 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 5a959618b9e9289355531d0e0306b9a4: Waiting for close lock at 1676174552630Disabling compacts and flushes for region at 1676174552630Disabling writes for close at 1676174552631 (+1 ms)Writing region close event to WAL at 1676174552631Closed at 1676174552631 2023-02-12 04:02:32,633 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 04:02:32,634 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174552634"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174552634"}]},"ts":"1676174552634"} 2023-02-12 04:02:32,636 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 04:02:32,637 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 04:02:32,637 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174552637"}]},"ts":"1676174552637"} 2023-02-12 04:02:32,638 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-12 04:02:32,641 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-12 04:02:32,642 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-12 04:02:32,642 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-12 04:02:32,642 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-12 04:02:32,642 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-12 04:02:32,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5a959618b9e9289355531d0e0306b9a4, ASSIGN}] 2023-02-12 04:02:32,644 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5a959618b9e9289355531d0e0306b9a4, ASSIGN 2023-02-12 04:02:32,645 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5a959618b9e9289355531d0e0306b9a4, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,40151,1676174449139; forceNewPlan=false, retain=false 2023-02-12 04:02:32,795 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-12 04:02:32,798 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=5a959618b9e9289355531d0e0306b9a4, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:32,798 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174552797"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174552797"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174552797"}]},"ts":"1676174552797"} 2023-02-12 04:02:32,801 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 5a959618b9e9289355531d0e0306b9a4, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:02:32,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:32,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5a959618b9e9289355531d0e0306b9a4, NAME => 'testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:02:32,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:02:32,963 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:32,963 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:32,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:32,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:32,966 INFO [StoreOpener-5a959618b9e9289355531d0e0306b9a4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:32,969 DEBUG [StoreOpener-5a959618b9e9289355531d0e0306b9a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1 2023-02-12 04:02:32,969 DEBUG [StoreOpener-5a959618b9e9289355531d0e0306b9a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1 2023-02-12 04:02:32,970 INFO [StoreOpener-5a959618b9e9289355531d0e0306b9a4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5a959618b9e9289355531d0e0306b9a4 columnFamilyName cf1 2023-02-12 04:02:32,971 INFO [StoreOpener-5a959618b9e9289355531d0e0306b9a4-1] regionserver.HStore(310): Store=5a959618b9e9289355531d0e0306b9a4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:32,971 INFO [StoreOpener-5a959618b9e9289355531d0e0306b9a4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:32,974 DEBUG [StoreOpener-5a959618b9e9289355531d0e0306b9a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2 2023-02-12 04:02:32,974 DEBUG [StoreOpener-5a959618b9e9289355531d0e0306b9a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2 2023-02-12 04:02:32,975 INFO [StoreOpener-5a959618b9e9289355531d0e0306b9a4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5a959618b9e9289355531d0e0306b9a4 columnFamilyName cf2 2023-02-12 04:02:32,976 INFO [StoreOpener-5a959618b9e9289355531d0e0306b9a4-1] regionserver.HStore(310): Store=5a959618b9e9289355531d0e0306b9a4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:32,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:32,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:32,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:02:32,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:32,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 04:02:32,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 5a959618b9e9289355531d0e0306b9a4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73703641, jitterRate=0.09826983511447906}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:02:32,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 5a959618b9e9289355531d0e0306b9a4: Running coprocessor pre-open hook at 1676174552964Writing region info on filesystem at 1676174552964Initializing all the Stores at 1676174552966 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174552966Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174552966Cleaning up temporary data from old regions at 1676174552979 (+13 ms)Cleaning up detritus from prior splits at 1676174552980 (+1 ms)Running coprocessor post-open hooks at 1676174552987 (+7 ms)Region opened successfully at 1676174552987 2023-02-12 04:02:32,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4., pid=127, masterSystemTime=1676174552956 2023-02-12 04:02:32,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:32,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:32,991 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=5a959618b9e9289355531d0e0306b9a4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:32,991 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174552991"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174552991"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174552991"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174552991"}]},"ts":"1676174552991"} 2023-02-12 04:02:32,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-12 04:02:32,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 5a959618b9e9289355531d0e0306b9a4, server=jenkins-hbase20.apache.org,40151,1676174449139 in 192 msec 2023-02-12 04:02:32,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-12 04:02:32,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5a959618b9e9289355531d0e0306b9a4, ASSIGN in 353 msec 2023-02-12 04:02:32,998 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 04:02:32,998 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174552998"}]},"ts":"1676174552998"} 2023-02-12 04:02:32,999 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-12 04:02:33,000 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 04:02:33,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 402 msec 2023-02-12 04:02:33,110 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:33,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,111 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-12 04:02:33,111 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,111 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-12 04:02:33,111 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-12 04:02:33,112 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,112 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,112 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,113 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,112 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,112 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,113 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,113 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:33,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,114 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,114 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,114 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,114 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,114 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,114 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,114 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,114 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,114 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,114 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,114 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,114 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,115 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,114 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:33,115 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:33,115 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:33,115 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. started... 2023-02-12 04:02:33,116 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a959618b9e9289355531d0e0306b9a4 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-12 04:02:33,116 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,116 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,116 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,116 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,116 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,116 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,123 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/003e43f6cc744f0fa358e68d077b211d 2023-02-12 04:02:33,134 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/8e0a9954852446d88f0efed3178f7f28 2023-02-12 04:02:33,138 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/003e43f6cc744f0fa358e68d077b211d as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/003e43f6cc744f0fa358e68d077b211d 2023-02-12 04:02:33,142 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/003e43f6cc744f0fa358e68d077b211d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:02:33,143 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/8e0a9954852446d88f0efed3178f7f28 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/8e0a9954852446d88f0efed3178f7f28 2023-02-12 04:02:33,147 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/8e0a9954852446d88f0efed3178f7f28, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:02:33,148 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 5a959618b9e9289355531d0e0306b9a4 in 32ms, sequenceid=6, compaction requested=false 2023-02-12 04:02:33,148 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-12 04:02:33,149 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a959618b9e9289355531d0e0306b9a4: 2023-02-12 04:02:33,149 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:33,149 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:33,149 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:33,149 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,149 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,149 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,150 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,150 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,151 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,151 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-12 04:02:33,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ee793c9[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:33,154 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-12 04:02:33,154 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,155 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,155 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,155 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,155 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,155 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,155 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:33,155 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,155 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:33,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,156 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,156 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,156 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,156 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,156 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:33,156 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:33,156 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,156 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,156 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,157 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,157 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,157 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,157 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,157 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,157 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,157 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:02:33,162 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:02:33,162 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-12 04:02:33,162 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:33,162 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:33,162 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,162 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:33,170 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,170 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,170 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,170 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,170 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,170 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,170 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,171 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,172 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,173 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,177 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,177 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,178 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,182 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,182 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,182 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,182 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,182 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,182 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,182 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,182 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,182 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,182 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-12 04:02:33,182 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:33,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,182 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,182 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,183 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,183 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,183 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,183 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,183 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:33,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,183 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,183 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:33,183 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,184 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,284 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:33,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:33,304 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:33,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:33,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,306 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-12 04:02:33,306 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,306 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-12 04:02:33,306 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-12 04:02:33,306 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,306 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,307 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,307 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,307 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,307 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,307 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,307 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,308 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,308 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:33,308 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,308 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,308 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,309 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,309 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. started... 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,309 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a959618b9e9289355531d0e0306b9a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,309 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,310 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,311 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,311 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,311 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,311 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,311 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,319 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/52c54a9eceda455b98f99825c7216e9d 2023-02-12 04:02:33,333 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/bbde44e6017442798ded03a83a1ac3ac 2023-02-12 04:02:33,339 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/52c54a9eceda455b98f99825c7216e9d as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/52c54a9eceda455b98f99825c7216e9d 2023-02-12 04:02:33,344 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/52c54a9eceda455b98f99825c7216e9d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:02:33,345 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/bbde44e6017442798ded03a83a1ac3ac as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/bbde44e6017442798ded03a83a1ac3ac 2023-02-12 04:02:33,351 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/bbde44e6017442798ded03a83a1ac3ac, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:02:33,352 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5a959618b9e9289355531d0e0306b9a4 in 43ms, sequenceid=19, compaction requested=false 2023-02-12 04:02:33,352 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a959618b9e9289355531d0e0306b9a4: 2023-02-12 04:02:33,352 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:33,353 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:33,353 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:33,353 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,353 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,353 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,354 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,354 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,355 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,355 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,358 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-12 04:02:33,358 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@748e2c01[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:33,358 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-12 04:02:33,358 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,359 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,359 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,359 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,360 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,360 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:33,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,360 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,360 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,360 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,360 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,360 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:33,360 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:33,360 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,360 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,360 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:33,361 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,361 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,361 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,362 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,362 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,362 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,362 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,361 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,362 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,362 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:02:33,364 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:02:33,364 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-12 04:02:33,364 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:33,364 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:33,364 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,364 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:33,365 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,365 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,365 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,365 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,365 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,365 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,365 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,365 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,366 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,366 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,367 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,367 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,368 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,369 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,369 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,369 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,369 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,369 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,369 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-12 04:02:33,369 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:33,369 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,369 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,369 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,369 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,369 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:33,369 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,370 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:33,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,370 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,370 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,370 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,370 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,370 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,370 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,370 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,370 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,470 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:33,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:33,490 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:33,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:33,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,491 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-12 04:02:33,491 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,491 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-12 04:02:33,491 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-12 04:02:33,492 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,492 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,492 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,492 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,492 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,492 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,493 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,493 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:33,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,494 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,494 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,494 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:33,494 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,494 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,494 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:33,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,494 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,494 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,494 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,494 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,494 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,494 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,494 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,494 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,494 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,494 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:33,494 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,495 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:33,495 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:33,495 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,495 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,495 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,495 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,495 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:33,495 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,495 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. started... 2023-02-12 04:02:33,495 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a959618b9e9289355531d0e0306b9a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:33,495 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,496 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,496 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,497 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,497 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,497 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,509 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/fd5fb670808b4535908bcd58a47c252e 2023-02-12 04:02:33,524 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/4c3f64db38e34b208d2ee5dd05a83d95 2023-02-12 04:02:33,528 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/fd5fb670808b4535908bcd58a47c252e as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/fd5fb670808b4535908bcd58a47c252e 2023-02-12 04:02:33,531 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/fd5fb670808b4535908bcd58a47c252e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:02:33,531 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/4c3f64db38e34b208d2ee5dd05a83d95 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/4c3f64db38e34b208d2ee5dd05a83d95 2023-02-12 04:02:33,534 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/4c3f64db38e34b208d2ee5dd05a83d95, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:02:33,535 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5a959618b9e9289355531d0e0306b9a4 in 40ms, sequenceid=32, compaction requested=true 2023-02-12 04:02:33,535 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a959618b9e9289355531d0e0306b9a4: 2023-02-12 04:02:33,535 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:33,535 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:33,535 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:33,535 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,536 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,536 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,537 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,537 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,537 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,537 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-12 04:02:33,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77ecf09f[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:33,538 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-12 04:02:33,538 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,539 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,539 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,539 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,539 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,539 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,539 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,539 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,539 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,539 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:33,539 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,539 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:33,539 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:33,539 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,540 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:33,540 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,540 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,540 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,540 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,540 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,540 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,540 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,540 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,540 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,541 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:02:33,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:02:33,543 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-12 04:02:33,543 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:33,543 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:33,543 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,543 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:33,544 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,544 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,544 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,544 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,544 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,544 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,544 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,545 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,545 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,545 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,546 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,546 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,547 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,548 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,548 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,548 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,549 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,548 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,549 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,549 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,549 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-12 04:02:33,549 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:33,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,549 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,549 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,549 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,549 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,549 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,549 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,549 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:33,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,549 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:33,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,549 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,550 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,550 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,550 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,550 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,650 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:33,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:33,665 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:33,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:33,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,668 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-12 04:02:33,668 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,668 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-12 04:02:33,668 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-12 04:02:33,668 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,668 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,669 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,669 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,669 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,669 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,669 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,669 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:33,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,670 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,670 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,670 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,670 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,671 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,671 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:33,671 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,671 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,671 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,671 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,671 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,671 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:33,671 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:33,672 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:33,672 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. started... 2023-02-12 04:02:33,672 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a959618b9e9289355531d0e0306b9a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:33,672 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,672 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,672 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,672 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,672 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,672 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,679 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/0f8122fd363044d4829ea3e19b6c6efa 2023-02-12 04:02:33,692 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/fd6be7700a98450b909550a446ee039c 2023-02-12 04:02:33,697 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/0f8122fd363044d4829ea3e19b6c6efa as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/0f8122fd363044d4829ea3e19b6c6efa 2023-02-12 04:02:33,703 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/0f8122fd363044d4829ea3e19b6c6efa, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:02:33,704 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/fd6be7700a98450b909550a446ee039c as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/fd6be7700a98450b909550a446ee039c 2023-02-12 04:02:33,709 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/fd6be7700a98450b909550a446ee039c, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:02:33,710 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5a959618b9e9289355531d0e0306b9a4 in 38ms, sequenceid=45, compaction requested=true 2023-02-12 04:02:33,710 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a959618b9e9289355531d0e0306b9a4: 2023-02-12 04:02:33,710 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:33,710 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:33,710 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:33,710 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,710 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,710 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,712 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,712 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,712 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,712 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,715 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-12 04:02:33,715 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c375fef[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:33,715 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-12 04:02:33,715 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,716 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,716 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,716 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,716 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:33,716 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,716 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,716 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,716 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,716 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:33,716 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,717 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:33,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,717 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,717 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,717 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:33,717 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,717 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,717 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,718 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,718 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,718 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,718 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,718 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,718 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,718 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:02:33,722 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:02:33,722 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-12 04:02:33,722 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:33,722 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:33,722 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,722 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:33,723 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,723 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,723 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,723 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,723 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,724 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,723 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,724 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,725 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,726 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,728 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,728 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,729 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,733 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,733 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,733 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,733 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,733 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,733 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-12 04:02:33,733 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:33,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,733 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,733 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,733 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,733 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,734 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:33,734 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:33,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,734 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,733 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,734 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,734 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,734 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,734 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,734 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,734 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,735 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,834 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:33,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:33,854 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:33,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:33,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,856 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-12 04:02:33,856 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,856 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-12 04:02:33,857 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-12 04:02:33,857 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,857 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,858 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,858 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,858 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,858 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,859 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,859 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:33,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,859 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,859 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,859 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,859 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:33,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,860 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,860 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,860 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:33,860 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,860 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:33,860 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,860 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,860 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:33,860 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,860 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,860 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,860 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,860 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,860 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:33,860 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:33,860 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,861 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,860 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,860 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:33,861 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,861 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,861 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:33,861 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:33,861 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:33,861 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. started... 2023-02-12 04:02:33,861 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a959618b9e9289355531d0e0306b9a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:33,862 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,862 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,862 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,862 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,862 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,862 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,871 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/f6a4775b39fd4b778b3b2e7eedcfd32a 2023-02-12 04:02:33,884 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/5b666f1ec2584caa92064e3076d3024e 2023-02-12 04:02:33,887 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/f6a4775b39fd4b778b3b2e7eedcfd32a as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/f6a4775b39fd4b778b3b2e7eedcfd32a 2023-02-12 04:02:33,891 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/f6a4775b39fd4b778b3b2e7eedcfd32a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:02:33,892 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/5b666f1ec2584caa92064e3076d3024e as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/5b666f1ec2584caa92064e3076d3024e 2023-02-12 04:02:33,896 ERROR [BP-664456210-148.251.75.209-1676174445046 heartbeating to localhost.localdomain/127.0.0.1:37867] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-664456210-148.251.75.209-1676174445046 (Datanode Uuid 6bee6fa0-f002-488e-898d-32eb09388f7e) service to localhost.localdomain/127.0.0.1:37867 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-12 04:02:33,897 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/5b666f1ec2584caa92064e3076d3024e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:02:33,897 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5a959618b9e9289355531d0e0306b9a4 in 36ms, sequenceid=58, compaction requested=true 2023-02-12 04:02:33,897 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a959618b9e9289355531d0e0306b9a4: 2023-02-12 04:02:33,897 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:33,898 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:33,898 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:33,898 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,898 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:33,898 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:33,899 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,899 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,899 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,899 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:33,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,901 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-12 04:02:33,901 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e8cfa7b[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:33,901 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-12 04:02:33,901 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,902 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,902 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,902 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,902 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,902 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,902 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:33,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,902 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,902 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,902 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:33,902 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:33,902 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,902 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:33,902 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:33,903 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:33,903 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,903 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,903 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,903 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,903 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,904 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,904 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:33,904 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,904 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,904 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:33,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:02:33,907 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:02:33,907 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-12 04:02:33,907 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:33,907 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:33,907 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,907 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:33,908 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,908 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,908 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,908 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,908 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:33,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,908 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,908 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:33,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,908 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,909 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:33,909 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:33,910 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,911 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,911 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,913 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,913 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,913 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,913 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,913 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-12 04:02:33,913 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:33,913 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:33,913 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,913 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:33,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,913 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,913 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:33,913 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:33,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:33,913 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:33,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,913 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,913 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:33,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:33,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,914 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:33,914 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:33,914 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,914 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:33,914 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:33,914 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:33,914 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,014 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:34,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:34,026 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:34,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:34,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,028 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-12 04:02:34,028 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:34,028 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-12 04:02:34,028 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-12 04:02:34,029 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,029 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,030 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:34,030 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:34,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:34,030 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:34,030 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:34,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:34,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:34,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:34,030 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,030 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:34,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:34,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:34,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:34,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,031 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,031 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:34,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,032 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:34,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:34,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,032 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:34,032 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:34,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-12 04:02:34,032 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:34,032 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:34,032 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:34,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,032 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:34,032 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:34,032 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:34,033 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:34,033 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:34,032 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:34,032 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-12 04:02:34,033 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:34,033 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. started... 2023-02-12 04:02:34,033 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-12 04:02:34,033 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:34,033 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:34,033 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a959618b9e9289355531d0e0306b9a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:34,035 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,035 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,035 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,035 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:34,036 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,036 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:34,045 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/639bb2a7e755484888ca19bb96c4bc29 2023-02-12 04:02:34,046 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 04:02:34,057 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/7c05c61737b5489f9343861721020aa4 2023-02-12 04:02:34,062 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/639bb2a7e755484888ca19bb96c4bc29 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/639bb2a7e755484888ca19bb96c4bc29 2023-02-12 04:02:34,065 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/639bb2a7e755484888ca19bb96c4bc29, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:02:34,065 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/7c05c61737b5489f9343861721020aa4 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/7c05c61737b5489f9343861721020aa4 2023-02-12 04:02:34,068 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/7c05c61737b5489f9343861721020aa4, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:02:34,069 INFO [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5a959618b9e9289355531d0e0306b9a4 in 36ms, sequenceid=71, compaction requested=true 2023-02-12 04:02:34,069 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a959618b9e9289355531d0e0306b9a4: 2023-02-12 04:02:34,069 DEBUG [rs(jenkins-hbase20.apache.org,40151,1676174449139)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:34,069 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:34,069 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:34,069 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,069 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-12 04:02:34,069 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-12 04:02:34,070 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,070 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:34,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:34,071 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,071 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:34,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:34,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:34,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:34,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:34,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:34,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-12 04:02:34,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b672f8f[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:34,072 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-12 04:02:34,072 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,073 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,073 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,073 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,073 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,073 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:34,073 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:34,073 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:34,073 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:34,073 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-12 04:02:34,073 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:34,073 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:34,073 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-12 04:02:34,073 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:34,074 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:34,074 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:34,074 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:34,074 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:34,074 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:34,074 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:34,074 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,074 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-12 04:02:34,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:34,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:34,074 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:34,074 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:34,075 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-12 04:02:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:34,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:34,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:34,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:34,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,40151,1676174449139': 2023-02-12 04:02:34,077 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-12 04:02:34,077 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-12 04:02:34,078 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:34,078 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:34,078 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,078 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:34,078 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,079 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,079 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:34,079 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,079 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:34,078 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:34,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,079 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:34,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:34,079 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:34,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:34,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:34,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,080 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:34,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,080 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:34,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:34,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:34,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,081 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:34,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:34,081 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:34,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:34,082 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,083 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,083 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:34,083 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:34,084 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:34,083 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:34,084 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:34,084 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:34,083 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:34,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:34,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:34,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:34,084 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:34,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:34,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-12 04:02:34,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:34,084 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:34,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:34,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:34,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:34,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:34,084 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:34,084 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:34,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:34,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:34,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:34,084 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:34,084 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,085 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,085 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,085 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:34,085 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:34,085 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,085 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-12 04:02:34,185 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:34,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:34,196 DEBUG [Listener at localhost.localdomain/35207] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,197 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=5a959618b9e9289355531d0e0306b9a4, daughterA=e7e2ef82c4b8ad18ce86c0da27ce9ce4, daughterB=fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:34,198 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=5a959618b9e9289355531d0e0306b9a4, daughterA=e7e2ef82c4b8ad18ce86c0da27ce9ce4, daughterB=fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:34,198 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=5a959618b9e9289355531d0e0306b9a4, daughterA=e7e2ef82c4b8ad18ce86c0da27ce9ce4, daughterB=fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:34,198 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=5a959618b9e9289355531d0e0306b9a4, daughterA=e7e2ef82c4b8ad18ce86c0da27ce9ce4, daughterB=fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:34,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5a959618b9e9289355531d0e0306b9a4, UNASSIGN}] 2023-02-12 04:02:34,202 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5a959618b9e9289355531d0e0306b9a4, UNASSIGN 2023-02-12 04:02:34,202 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=5a959618b9e9289355531d0e0306b9a4, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,203 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174554202"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174554202"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174554202"}]},"ts":"1676174554202"} 2023-02-12 04:02:34,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 5a959618b9e9289355531d0e0306b9a4, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:02:34,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 5a959618b9e9289355531d0e0306b9a4, disabling compactions & flushes 2023-02-12 04:02:34,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:34,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:34,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. after waiting 0 ms 2023-02-12 04:02:34,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:34,358 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 5a959618b9e9289355531d0e0306b9a4 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-12 04:02:34,375 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/6a525ba3cad24446b098490cad7e5a9a 2023-02-12 04:02:34,386 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/09ce1f56d25c40889ef59ed29ddaa1db 2023-02-12 04:02:34,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf1/6a525ba3cad24446b098490cad7e5a9a as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/6a525ba3cad24446b098490cad7e5a9a 2023-02-12 04:02:34,397 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/6a525ba3cad24446b098490cad7e5a9a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:02:34,398 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.tmp/cf2/09ce1f56d25c40889ef59ed29ddaa1db as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/09ce1f56d25c40889ef59ed29ddaa1db 2023-02-12 04:02:34,404 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/09ce1f56d25c40889ef59ed29ddaa1db, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:02:34,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 5a959618b9e9289355531d0e0306b9a4 in 48ms, sequenceid=82, compaction requested=true 2023-02-12 04:02:34,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-12 04:02:34,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:34,410 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. 2023-02-12 04:02:34,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 5a959618b9e9289355531d0e0306b9a4: Waiting for close lock at 1676174554357Running coprocessor pre-close hooks at 1676174554357Disabling compacts and flushes for region at 1676174554357Disabling writes for close at 1676174554357Obtaining lock to block concurrent updates at 1676174554358 (+1 ms)Preparing flush snapshotting stores in 5a959618b9e9289355531d0e0306b9a4 at 1676174554358Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676174554358Flushing stores of testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. at 1676174554359 (+1 ms)Flushing 5a959618b9e9289355531d0e0306b9a4/cf1: creating writer at 1676174554364 (+5 ms)Flushing 5a959618b9e9289355531d0e0306b9a4/cf1: appending metadata at 1676174554366 (+2 ms)Flushing 5a959618b9e9289355531d0e0306b9a4/cf1: closing flushed file at 1676174554366Flushing 5a959618b9e9289355531d0e0306b9a4/cf2: creating writer at 1676174554378 (+12 ms)Flushing 5a959618b9e9289355531d0e0306b9a4/cf2: appending metadata at 1676174554380 (+2 ms)Flushing 5a959618b9e9289355531d0e0306b9a4/cf2: closing flushed file at 1676174554380Flushing 5a959618b9e9289355531d0e0306b9a4/cf1: reopening flushed file at 1676174554393 (+13 ms)Flushing 5a959618b9e9289355531d0e0306b9a4/cf2: reopening flushed file at 1676174554399 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 5a959618b9e9289355531d0e0306b9a4 in 48ms, sequenceid=82, compaction requested=true at 1676174554405 (+6 ms)Writing region close event to WAL at 1676174554406 (+1 ms)Running coprocessor post-close hooks at 1676174554409 (+3 ms)Closed at 1676174554410 (+1 ms) 2023-02-12 04:02:34,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,412 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=5a959618b9e9289355531d0e0306b9a4, regionState=CLOSED 2023-02-12 04:02:34,412 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174554412"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174554412"}]},"ts":"1676174554412"} 2023-02-12 04:02:34,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-12 04:02:34,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 5a959618b9e9289355531d0e0306b9a4, server=jenkins-hbase20.apache.org,40151,1676174449139 in 210 msec 2023-02-12 04:02:34,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-12 04:02:34,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=5a959618b9e9289355531d0e0306b9a4, UNASSIGN in 214 msec 2023-02-12 04:02:34,426 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=5a959618b9e9289355531d0e0306b9a4, threads=14 2023-02-12 04:02:34,427 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/09ce1f56d25c40889ef59ed29ddaa1db for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,427 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/4c3f64db38e34b208d2ee5dd05a83d95 for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,427 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/5b666f1ec2584caa92064e3076d3024e for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,427 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/7c05c61737b5489f9343861721020aa4 for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,427 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/8e0a9954852446d88f0efed3178f7f28 for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,427 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/bbde44e6017442798ded03a83a1ac3ac for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,427 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/fd6be7700a98450b909550a446ee039c for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,427 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/003e43f6cc744f0fa358e68d077b211d for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,427 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/0f8122fd363044d4829ea3e19b6c6efa for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,427 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/52c54a9eceda455b98f99825c7216e9d for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,427 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/639bb2a7e755484888ca19bb96c4bc29 for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,427 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/6a525ba3cad24446b098490cad7e5a9a for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,428 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/f6a4775b39fd4b778b3b2e7eedcfd32a for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,428 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/fd5fb670808b4535908bcd58a47c252e for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,431 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/7c05c61737b5489f9343861721020aa4, top=false 2023-02-12 04:02:34,431 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/0f8122fd363044d4829ea3e19b6c6efa, top=false 2023-02-12 04:02:34,433 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/5b666f1ec2584caa92064e3076d3024e, top=false 2023-02-12 04:02:34,433 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/52c54a9eceda455b98f99825c7216e9d, top=false 2023-02-12 04:02:34,434 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/003e43f6cc744f0fa358e68d077b211d, top=false 2023-02-12 04:02:34,439 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-003e43f6cc744f0fa358e68d077b211d for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,439 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/09ce1f56d25c40889ef59ed29ddaa1db, top=false 2023-02-12 04:02:34,440 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/fd5fb670808b4535908bcd58a47c252e, top=false 2023-02-12 04:02:34,441 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/fd6be7700a98450b909550a446ee039c, top=false 2023-02-12 04:02:34,441 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/f6a4775b39fd4b778b3b2e7eedcfd32a, top=false 2023-02-12 04:02:34,443 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-5b666f1ec2584caa92064e3076d3024e for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,443 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/4c3f64db38e34b208d2ee5dd05a83d95, top=false 2023-02-12 04:02:34,444 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/8e0a9954852446d88f0efed3178f7f28, top=false 2023-02-12 04:02:34,444 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/6a525ba3cad24446b098490cad7e5a9a, top=false 2023-02-12 04:02:34,445 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/bbde44e6017442798ded03a83a1ac3ac, top=false 2023-02-12 04:02:34,446 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/003e43f6cc744f0fa358e68d077b211d for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,447 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-09ce1f56d25c40889ef59ed29ddaa1db for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,447 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/639bb2a7e755484888ca19bb96c4bc29, top=false 2023-02-12 04:02:34,448 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-0f8122fd363044d4829ea3e19b6c6efa for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,448 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd5fb670808b4535908bcd58a47c252e for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,449 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-7c05c61737b5489f9343861721020aa4 for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,449 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/5b666f1ec2584caa92064e3076d3024e for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,451 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-52c54a9eceda455b98f99825c7216e9d for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,453 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/09ce1f56d25c40889ef59ed29ddaa1db for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,455 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/fd5fb670808b4535908bcd58a47c252e for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,455 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/52c54a9eceda455b98f99825c7216e9d for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,455 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/7c05c61737b5489f9343861721020aa4 for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,459 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/0f8122fd363044d4829ea3e19b6c6efa for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,460 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd6be7700a98450b909550a446ee039c for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,462 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-4c3f64db38e34b208d2ee5dd05a83d95 for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,462 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-bbde44e6017442798ded03a83a1ac3ac for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,462 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-f6a4775b39fd4b778b3b2e7eedcfd32a for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,463 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/fd6be7700a98450b909550a446ee039c for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,463 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-8e0a9954852446d88f0efed3178f7f28 for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,463 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-6a525ba3cad24446b098490cad7e5a9a for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,464 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/.splits/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-639bb2a7e755484888ca19bb96c4bc29 for child: e7e2ef82c4b8ad18ce86c0da27ce9ce4, parent: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,465 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/4c3f64db38e34b208d2ee5dd05a83d95 for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,465 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/f6a4775b39fd4b778b3b2e7eedcfd32a for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,466 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/8e0a9954852446d88f0efed3178f7f28 for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,466 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/6a525ba3cad24446b098490cad7e5a9a for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,467 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/639bb2a7e755484888ca19bb96c4bc29 for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,468 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/bbde44e6017442798ded03a83a1ac3ac for region: 5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:34,468 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 5a959618b9e9289355531d0e0306b9a4 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-12 04:02:34,493 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-12 04:02:34,494 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-12 04:02:34,495 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676174554495"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676174554495"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676174554495"}]},"ts":"1676174554495"} 2023-02-12 04:02:34,496 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676174554495"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174554495"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174554495"}]},"ts":"1676174554495"} 2023-02-12 04:02:34,496 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676174554495"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174554495"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174554495"}]},"ts":"1676174554495"} 2023-02-12 04:02:34,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e7e2ef82c4b8ad18ce86c0da27ce9ce4, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=fd173b0b8daf9a032ee7db30a34f3aae, ASSIGN}] 2023-02-12 04:02:34,499 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=fd173b0b8daf9a032ee7db30a34f3aae, ASSIGN 2023-02-12 04:02:34,500 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=fd173b0b8daf9a032ee7db30a34f3aae, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,40151,1676174449139; forceNewPlan=false, retain=false 2023-02-12 04:02:34,500 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e7e2ef82c4b8ad18ce86c0da27ce9ce4, ASSIGN 2023-02-12 04:02:34,500 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e7e2ef82c4b8ad18ce86c0da27ce9ce4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,40151,1676174449139; forceNewPlan=false, retain=false 2023-02-12 04:02:34,650 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-12 04:02:34,652 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=fd173b0b8daf9a032ee7db30a34f3aae, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,652 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676174554652"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174554652"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174554652"}]},"ts":"1676174554652"} 2023-02-12 04:02:34,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure fd173b0b8daf9a032ee7db30a34f3aae, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:02:34,659 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=e7e2ef82c4b8ad18ce86c0da27ce9ce4, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,659 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676174554659"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174554659"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174554659"}]},"ts":"1676174554659"} 2023-02-12 04:02:34,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure e7e2ef82c4b8ad18ce86c0da27ce9ce4, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:02:34,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae. 2023-02-12 04:02:34,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => fd173b0b8daf9a032ee7db30a34f3aae, NAME => 'testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae.', STARTKEY => '71', ENDKEY => ''} 2023-02-12 04:02:34,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:02:34,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:34,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:34,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:34,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:34,814 INFO [StoreOpener-fd173b0b8daf9a032ee7db30a34f3aae-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:34,816 DEBUG [StoreOpener-fd173b0b8daf9a032ee7db30a34f3aae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae/cf1 2023-02-12 04:02:34,817 DEBUG [StoreOpener-fd173b0b8daf9a032ee7db30a34f3aae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae/cf1 2023-02-12 04:02:34,817 INFO [StoreOpener-fd173b0b8daf9a032ee7db30a34f3aae-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fd173b0b8daf9a032ee7db30a34f3aae columnFamilyName cf1 2023-02-12 04:02:34,818 INFO [StoreOpener-fd173b0b8daf9a032ee7db30a34f3aae-1] regionserver.HStore(310): Store=fd173b0b8daf9a032ee7db30a34f3aae/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:34,818 INFO [StoreOpener-fd173b0b8daf9a032ee7db30a34f3aae-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:34,821 DEBUG [StoreOpener-fd173b0b8daf9a032ee7db30a34f3aae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae/cf2 2023-02-12 04:02:34,821 DEBUG [StoreOpener-fd173b0b8daf9a032ee7db30a34f3aae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae/cf2 2023-02-12 04:02:34,822 INFO [StoreOpener-fd173b0b8daf9a032ee7db30a34f3aae-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fd173b0b8daf9a032ee7db30a34f3aae columnFamilyName cf2 2023-02-12 04:02:34,822 INFO [StoreOpener-fd173b0b8daf9a032ee7db30a34f3aae-1] regionserver.HStore(310): Store=fd173b0b8daf9a032ee7db30a34f3aae/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:34,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:34,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:34,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:02:34,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:34,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened fd173b0b8daf9a032ee7db30a34f3aae; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69430333, jitterRate=0.034592583775520325}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:02:34,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for fd173b0b8daf9a032ee7db30a34f3aae: Running coprocessor pre-open hook at 1676174554812Writing region info on filesystem at 1676174554812Initializing all the Stores at 1676174554813 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174554813Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174554814 (+1 ms)Cleaning up temporary data from old regions at 1676174554827 (+13 ms)Cleaning up detritus from prior splits at 1676174554827Running coprocessor post-open hooks at 1676174554831 (+4 ms)Region opened successfully at 1676174554831 2023-02-12 04:02:34,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae., pid=133, masterSystemTime=1676174554807 2023-02-12 04:02:34,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae. 2023-02-12 04:02:34,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae. 2023-02-12 04:02:34,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4. 2023-02-12 04:02:34,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e7e2ef82c4b8ad18ce86c0da27ce9ce4, NAME => 'testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.', STARTKEY => '', ENDKEY => '71'} 2023-02-12 04:02:34,834 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=fd173b0b8daf9a032ee7db30a34f3aae, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:02:34,834 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676174554834"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174554834"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174554834"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174554834"}]},"ts":"1676174554834"} 2023-02-12 04:02:34,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter e7e2ef82c4b8ad18ce86c0da27ce9ce4 2023-02-12 04:02:34,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:34,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for e7e2ef82c4b8ad18ce86c0da27ce9ce4 2023-02-12 04:02:34,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for e7e2ef82c4b8ad18ce86c0da27ce9ce4 2023-02-12 04:02:34,836 INFO [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e7e2ef82c4b8ad18ce86c0da27ce9ce4 2023-02-12 04:02:34,837 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1 2023-02-12 04:02:34,837 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1 2023-02-12 04:02:34,838 INFO [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e7e2ef82c4b8ad18ce86c0da27ce9ce4 columnFamilyName cf1 2023-02-12 04:02:34,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-12 04:02:34,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure fd173b0b8daf9a032ee7db30a34f3aae, server=jenkins-hbase20.apache.org,40151,1676174449139 in 181 msec 2023-02-12 04:02:34,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=fd173b0b8daf9a032ee7db30a34f3aae, ASSIGN in 340 msec 2023-02-12 04:02:34,846 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-003e43f6cc744f0fa358e68d077b211d 2023-02-12 04:02:34,848 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-0f8122fd363044d4829ea3e19b6c6efa 2023-02-12 04:02:34,851 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-52c54a9eceda455b98f99825c7216e9d 2023-02-12 04:02:34,853 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-639bb2a7e755484888ca19bb96c4bc29 2023-02-12 04:02:34,855 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-6a525ba3cad24446b098490cad7e5a9a 2023-02-12 04:02:34,857 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-f6a4775b39fd4b778b3b2e7eedcfd32a 2023-02-12 04:02:34,859 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd5fb670808b4535908bcd58a47c252e 2023-02-12 04:02:34,860 INFO [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(310): Store=e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:34,860 INFO [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e7e2ef82c4b8ad18ce86c0da27ce9ce4 2023-02-12 04:02:34,860 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2 2023-02-12 04:02:34,860 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2 2023-02-12 04:02:34,861 INFO [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e7e2ef82c4b8ad18ce86c0da27ce9ce4 columnFamilyName cf2 2023-02-12 04:02:34,864 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-09ce1f56d25c40889ef59ed29ddaa1db 2023-02-12 04:02:34,866 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-4c3f64db38e34b208d2ee5dd05a83d95 2023-02-12 04:02:34,869 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-5b666f1ec2584caa92064e3076d3024e 2023-02-12 04:02:34,872 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-7c05c61737b5489f9343861721020aa4 2023-02-12 04:02:34,874 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-8e0a9954852446d88f0efed3178f7f28 2023-02-12 04:02:34,877 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-bbde44e6017442798ded03a83a1ac3ac 2023-02-12 04:02:34,881 DEBUG [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd6be7700a98450b909550a446ee039c 2023-02-12 04:02:34,881 INFO [StoreOpener-e7e2ef82c4b8ad18ce86c0da27ce9ce4-1] regionserver.HStore(310): Store=e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:34,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4 2023-02-12 04:02:34,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4 2023-02-12 04:02:34,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:02:34,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for e7e2ef82c4b8ad18ce86c0da27ce9ce4 2023-02-12 04:02:34,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened e7e2ef82c4b8ad18ce86c0da27ce9ce4; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66747533, jitterRate=-0.0053842514753341675}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:02:34,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for e7e2ef82c4b8ad18ce86c0da27ce9ce4: Running coprocessor pre-open hook at 1676174554835Writing region info on filesystem at 1676174554835Initializing all the Stores at 1676174554836 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174554836Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174554836Cleaning up temporary data from old regions at 1676174554883 (+47 ms)Cleaning up detritus from prior splits at 1676174554883Running coprocessor post-open hooks at 1676174554886 (+3 ms)Region opened successfully at 1676174554886 2023-02-12 04:02:34,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4., pid=134, masterSystemTime=1676174554807 2023-02-12 04:02:34,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:34,887 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:02:34,887 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:02:34,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:02:34,887 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-12 04:02:34,888 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-12 04:02:34,888 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1912): e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1 is initiating minor compaction (all files) 2023-02-12 04:02:34,888 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1912): e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2 is initiating minor compaction (all files) 2023-02-12 04:02:34,888 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2 in testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4. 2023-02-12 04:02:34,888 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1 in testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4. 2023-02-12 04:02:34,888 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-8e0a9954852446d88f0efed3178f7f28, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-bbde44e6017442798ded03a83a1ac3ac, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-4c3f64db38e34b208d2ee5dd05a83d95, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd6be7700a98450b909550a446ee039c, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-5b666f1ec2584caa92064e3076d3024e, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-7c05c61737b5489f9343861721020aa4, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-09ce1f56d25c40889ef59ed29ddaa1db] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/.tmp, totalSize=34.9 K 2023-02-12 04:02:34,888 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-003e43f6cc744f0fa358e68d077b211d, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-52c54a9eceda455b98f99825c7216e9d, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd5fb670808b4535908bcd58a47c252e, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-0f8122fd363044d4829ea3e19b6c6efa, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-f6a4775b39fd4b778b3b2e7eedcfd32a, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-639bb2a7e755484888ca19bb96c4bc29, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-6a525ba3cad24446b098490cad7e5a9a] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/.tmp, totalSize=34.9 K 2023-02-12 04:02:34,888 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-8e0a9954852446d88f0efed3178f7f28, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174553101 2023-02-12 04:02:34,888 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-003e43f6cc744f0fa358e68d077b211d, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174553101 2023-02-12 04:02:34,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4. 2023-02-12 04:02:34,889 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-52c54a9eceda455b98f99825c7216e9d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174553287 2023-02-12 04:02:34,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4. 2023-02-12 04:02:34,889 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-bbde44e6017442798ded03a83a1ac3ac, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174553287 2023-02-12 04:02:34,889 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd5fb670808b4535908bcd58a47c252e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174553473 2023-02-12 04:02:34,890 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-4c3f64db38e34b208d2ee5dd05a83d95, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174553473 2023-02-12 04:02:34,890 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=e7e2ef82c4b8ad18ce86c0da27ce9ce4, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:34,890 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-0f8122fd363044d4829ea3e19b6c6efa, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174553651 2023-02-12 04:02:34,890 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676174554890"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174554890"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174554890"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174554890"}]},"ts":"1676174554890"} 2023-02-12 04:02:34,890 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd6be7700a98450b909550a446ee039c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174553651 2023-02-12 04:02:34,890 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-f6a4775b39fd4b778b3b2e7eedcfd32a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174553837 2023-02-12 04:02:34,891 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-5b666f1ec2584caa92064e3076d3024e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174553837 2023-02-12 04:02:34,891 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-639bb2a7e755484888ca19bb96c4bc29, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174554015 2023-02-12 04:02:34,891 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-7c05c61737b5489f9343861721020aa4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174554015 2023-02-12 04:02:34,892 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-6a525ba3cad24446b098490cad7e5a9a, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676174554186 2023-02-12 04:02:34,892 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-09ce1f56d25c40889ef59ed29ddaa1db, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676174554186 2023-02-12 04:02:34,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-12 04:02:34,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure e7e2ef82c4b8ad18ce86c0da27ce9ce4, server=jenkins-hbase20.apache.org,40151,1676174449139 in 230 msec 2023-02-12 04:02:34,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-02-12 04:02:34,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e7e2ef82c4b8ad18ce86c0da27ce9ce4, ASSIGN in 398 msec 2023-02-12 04:02:34,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=5a959618b9e9289355531d0e0306b9a4, daughterA=e7e2ef82c4b8ad18ce86c0da27ce9ce4, daughterB=fd173b0b8daf9a032ee7db30a34f3aae in 702 msec 2023-02-12 04:02:34,907 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] throttle.PressureAwareThroughputController(145): e7e2ef82c4b8ad18ce86c0da27ce9ce4#cf1#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:02:34,907 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] throttle.PressureAwareThroughputController(145): e7e2ef82c4b8ad18ce86c0da27ce9ce4#cf2#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:02:34,921 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/.tmp/cf1/ebd15ba27f9e4d779cd06f2aff9f84a8 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/ebd15ba27f9e4d779cd06f2aff9f84a8 2023-02-12 04:02:34,923 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/.tmp/cf2/a374a0e8a5c341fe9324fe64da7275f3 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/a374a0e8a5c341fe9324fe64da7275f3 2023-02-12 04:02:34,925 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1 of e7e2ef82c4b8ad18ce86c0da27ce9ce4 into ebd15ba27f9e4d779cd06f2aff9f84a8(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:02:34,925 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e7e2ef82c4b8ad18ce86c0da27ce9ce4: 2023-02-12 04:02:34,925 INFO [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4., storeName=e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1, priority=9, startTime=1676174554886; duration=0sec 2023-02-12 04:02:34,925 DEBUG [RS:0;jenkins-hbase20:40151-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:34,926 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2 of e7e2ef82c4b8ad18ce86c0da27ce9ce4 into a374a0e8a5c341fe9324fe64da7275f3(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:02:34,926 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e7e2ef82c4b8ad18ce86c0da27ce9ce4: 2023-02-12 04:02:34,926 INFO [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4., storeName=e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2, priority=9, startTime=1676174554887; duration=0sec 2023-02-12 04:02:34,927 DEBUG [RS:0;jenkins-hbase20:40151-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:34,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40151] ipc.CallRunner(144): callId: 989 service: ClientService methodName: Scan size: 157 connection: 148.251.75.209:53720 deadline: 1676174614948, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4. is not online on jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:35,067 DEBUG [Listener at localhost.localdomain/35207] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-12 04:02:35,072 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:02:35,074 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-12 04:02:35,075 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testSplitTableRegionEmptyDaughter 2023-02-12 04:02:35,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-12 04:02:35,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-12 04:02:35,079 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174555079"}]},"ts":"1676174555079"} 2023-02-12 04:02:35,080 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-12 04:02:35,082 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-12 04:02:35,082 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e7e2ef82c4b8ad18ce86c0da27ce9ce4, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=fd173b0b8daf9a032ee7db30a34f3aae, UNASSIGN}] 2023-02-12 04:02:35,083 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=fd173b0b8daf9a032ee7db30a34f3aae, UNASSIGN 2023-02-12 04:02:35,084 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=fd173b0b8daf9a032ee7db30a34f3aae, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:35,084 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676174555084"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174555084"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174555084"}]},"ts":"1676174555084"} 2023-02-12 04:02:35,086 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure fd173b0b8daf9a032ee7db30a34f3aae, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:02:35,089 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e7e2ef82c4b8ad18ce86c0da27ce9ce4, UNASSIGN 2023-02-12 04:02:35,089 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=e7e2ef82c4b8ad18ce86c0da27ce9ce4, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:35,090 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676174555089"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174555089"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174555089"}]},"ts":"1676174555089"} 2023-02-12 04:02:35,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure e7e2ef82c4b8ad18ce86c0da27ce9ce4, server=jenkins-hbase20.apache.org,40151,1676174449139}] 2023-02-12 04:02:35,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-12 04:02:35,238 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close e7e2ef82c4b8ad18ce86c0da27ce9ce4 2023-02-12 04:02:35,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing e7e2ef82c4b8ad18ce86c0da27ce9ce4, disabling compactions & flushes 2023-02-12 04:02:35,239 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4. 2023-02-12 04:02:35,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4. 2023-02-12 04:02:35,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4. after waiting 0 ms 2023-02-12 04:02:35,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4. 2023-02-12 04:02:35,240 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-003e43f6cc744f0fa358e68d077b211d, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-52c54a9eceda455b98f99825c7216e9d, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd5fb670808b4535908bcd58a47c252e, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-0f8122fd363044d4829ea3e19b6c6efa, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-f6a4775b39fd4b778b3b2e7eedcfd32a, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-639bb2a7e755484888ca19bb96c4bc29, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-6a525ba3cad24446b098490cad7e5a9a] to archive 2023-02-12 04:02:35,242 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:02:35,245 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-003e43f6cc744f0fa358e68d077b211d to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-003e43f6cc744f0fa358e68d077b211d 2023-02-12 04:02:35,248 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-52c54a9eceda455b98f99825c7216e9d to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-52c54a9eceda455b98f99825c7216e9d 2023-02-12 04:02:35,251 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd5fb670808b4535908bcd58a47c252e to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd5fb670808b4535908bcd58a47c252e 2023-02-12 04:02:35,253 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-0f8122fd363044d4829ea3e19b6c6efa to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-0f8122fd363044d4829ea3e19b6c6efa 2023-02-12 04:02:35,254 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-f6a4775b39fd4b778b3b2e7eedcfd32a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-f6a4775b39fd4b778b3b2e7eedcfd32a 2023-02-12 04:02:35,256 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-639bb2a7e755484888ca19bb96c4bc29 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-639bb2a7e755484888ca19bb96c4bc29 2023-02-12 04:02:35,258 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-6a525ba3cad24446b098490cad7e5a9a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-6a525ba3cad24446b098490cad7e5a9a 2023-02-12 04:02:35,260 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-8e0a9954852446d88f0efed3178f7f28, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-bbde44e6017442798ded03a83a1ac3ac, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-4c3f64db38e34b208d2ee5dd05a83d95, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd6be7700a98450b909550a446ee039c, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-5b666f1ec2584caa92064e3076d3024e, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-7c05c61737b5489f9343861721020aa4, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-09ce1f56d25c40889ef59ed29ddaa1db] to archive 2023-02-12 04:02:35,260 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:02:35,262 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-8e0a9954852446d88f0efed3178f7f28 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-8e0a9954852446d88f0efed3178f7f28 2023-02-12 04:02:35,264 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-bbde44e6017442798ded03a83a1ac3ac to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-bbde44e6017442798ded03a83a1ac3ac 2023-02-12 04:02:35,265 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-4c3f64db38e34b208d2ee5dd05a83d95 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-4c3f64db38e34b208d2ee5dd05a83d95 2023-02-12 04:02:35,267 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd6be7700a98450b909550a446ee039c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-fd6be7700a98450b909550a446ee039c 2023-02-12 04:02:35,268 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-5b666f1ec2584caa92064e3076d3024e to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-5b666f1ec2584caa92064e3076d3024e 2023-02-12 04:02:35,270 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-7c05c61737b5489f9343861721020aa4 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-7c05c61737b5489f9343861721020aa4 2023-02-12 04:02:35,271 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-09ce1f56d25c40889ef59ed29ddaa1db to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/testSplitTableRegionEmptyDaughter=5a959618b9e9289355531d0e0306b9a4-09ce1f56d25c40889ef59ed29ddaa1db 2023-02-12 04:02:35,276 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-12 04:02:35,277 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:35,277 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4. 2023-02-12 04:02:35,277 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for e7e2ef82c4b8ad18ce86c0da27ce9ce4: Waiting for close lock at 1676174555239Running coprocessor pre-close hooks at 1676174555239Disabling compacts and flushes for region at 1676174555239Disabling writes for close at 1676174555239Writing region close event to WAL at 1676174555272 (+33 ms)Running coprocessor post-close hooks at 1676174555277 (+5 ms)Closed at 1676174555277 2023-02-12 04:02:35,278 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed e7e2ef82c4b8ad18ce86c0da27ce9ce4 2023-02-12 04:02:35,278 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:35,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing fd173b0b8daf9a032ee7db30a34f3aae, disabling compactions & flushes 2023-02-12 04:02:35,280 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae. 2023-02-12 04:02:35,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae. 2023-02-12 04:02:35,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae. after waiting 0 ms 2023-02-12 04:02:35,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae. 2023-02-12 04:02:35,280 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=e7e2ef82c4b8ad18ce86c0da27ce9ce4, regionState=CLOSED 2023-02-12 04:02:35,280 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676174555280"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174555280"}]},"ts":"1676174555280"} 2023-02-12 04:02:35,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-12 04:02:35,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure e7e2ef82c4b8ad18ce86c0da27ce9ce4, server=jenkins-hbase20.apache.org,40151,1676174449139 in 190 msec 2023-02-12 04:02:35,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-12 04:02:35,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:35,285 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae. 2023-02-12 04:02:35,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for fd173b0b8daf9a032ee7db30a34f3aae: Waiting for close lock at 1676174555279Running coprocessor pre-close hooks at 1676174555279Disabling compacts and flushes for region at 1676174555279Disabling writes for close at 1676174555280 (+1 ms)Writing region close event to WAL at 1676174555280Running coprocessor post-close hooks at 1676174555284 (+4 ms)Closed at 1676174555285 (+1 ms) 2023-02-12 04:02:35,285 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e7e2ef82c4b8ad18ce86c0da27ce9ce4, UNASSIGN in 201 msec 2023-02-12 04:02:35,286 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:35,287 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=fd173b0b8daf9a032ee7db30a34f3aae, regionState=CLOSED 2023-02-12 04:02:35,287 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676174555287"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174555287"}]},"ts":"1676174555287"} 2023-02-12 04:02:35,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-12 04:02:35,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure fd173b0b8daf9a032ee7db30a34f3aae, server=jenkins-hbase20.apache.org,40151,1676174449139 in 202 msec 2023-02-12 04:02:35,291 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-12 04:02:35,291 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=fd173b0b8daf9a032ee7db30a34f3aae, UNASSIGN in 208 msec 2023-02-12 04:02:35,292 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174555292"}]},"ts":"1676174555292"} 2023-02-12 04:02:35,293 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-12 04:02:35,294 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-12 04:02:35,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 220 msec 2023-02-12 04:02:35,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-12 04:02:35,382 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-12 04:02:35,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testSplitTableRegionEmptyDaughter 2023-02-12 04:02:35,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-12 04:02:35,388 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-12 04:02:35,389 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-12 04:02:35,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-12 04:02:35,396 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:35,396 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:35,396 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4 2023-02-12 04:02:35,399 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/recovered.edits] 2023-02-12 04:02:35,399 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae/recovered.edits] 2023-02-12 04:02:35,399 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/recovered.edits] 2023-02-12 04:02:35,401 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/003e43f6cc744f0fa358e68d077b211d to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/003e43f6cc744f0fa358e68d077b211d 2023-02-12 04:02:35,401 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/ebd15ba27f9e4d779cd06f2aff9f84a8 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf1/ebd15ba27f9e4d779cd06f2aff9f84a8 2023-02-12 04:02:35,402 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/0f8122fd363044d4829ea3e19b6c6efa to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/0f8122fd363044d4829ea3e19b6c6efa 2023-02-12 04:02:35,403 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae/recovered.edits/88.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae/recovered.edits/88.seqid 2023-02-12 04:02:35,403 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/a374a0e8a5c341fe9324fe64da7275f3 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/cf2/a374a0e8a5c341fe9324fe64da7275f3 2023-02-12 04:02:35,403 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/fd173b0b8daf9a032ee7db30a34f3aae 2023-02-12 04:02:35,403 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/52c54a9eceda455b98f99825c7216e9d to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/52c54a9eceda455b98f99825c7216e9d 2023-02-12 04:02:35,404 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/639bb2a7e755484888ca19bb96c4bc29 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/639bb2a7e755484888ca19bb96c4bc29 2023-02-12 04:02:35,405 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/6a525ba3cad24446b098490cad7e5a9a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/6a525ba3cad24446b098490cad7e5a9a 2023-02-12 04:02:35,405 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/recovered.edits/90.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4/recovered.edits/90.seqid 2023-02-12 04:02:35,405 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/e7e2ef82c4b8ad18ce86c0da27ce9ce4 2023-02-12 04:02:35,406 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/f6a4775b39fd4b778b3b2e7eedcfd32a to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/f6a4775b39fd4b778b3b2e7eedcfd32a 2023-02-12 04:02:35,406 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/fd5fb670808b4535908bcd58a47c252e to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf1/fd5fb670808b4535908bcd58a47c252e 2023-02-12 04:02:35,408 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/09ce1f56d25c40889ef59ed29ddaa1db to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/09ce1f56d25c40889ef59ed29ddaa1db 2023-02-12 04:02:35,409 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/4c3f64db38e34b208d2ee5dd05a83d95 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/4c3f64db38e34b208d2ee5dd05a83d95 2023-02-12 04:02:35,409 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/5b666f1ec2584caa92064e3076d3024e to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/5b666f1ec2584caa92064e3076d3024e 2023-02-12 04:02:35,410 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/7c05c61737b5489f9343861721020aa4 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/7c05c61737b5489f9343861721020aa4 2023-02-12 04:02:35,412 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/8e0a9954852446d88f0efed3178f7f28 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/8e0a9954852446d88f0efed3178f7f28 2023-02-12 04:02:35,413 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/bbde44e6017442798ded03a83a1ac3ac to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/bbde44e6017442798ded03a83a1ac3ac 2023-02-12 04:02:35,414 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/fd6be7700a98450b909550a446ee039c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/cf2/fd6be7700a98450b909550a446ee039c 2023-02-12 04:02:35,416 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/recovered.edits/85.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4/recovered.edits/85.seqid 2023-02-12 04:02:35,417 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegionEmptyDaughter/5a959618b9e9289355531d0e0306b9a4 2023-02-12 04:02:35,417 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-12 04:02:35,418 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-12 04:02:35,420 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-12 04:02:35,422 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-12 04:02:35,423 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-12 04:02:35,423 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-12 04:02:35,423 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174555423"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:35,423 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174555423"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:35,423 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174555423"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:35,425 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-12 04:02:35,425 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 5a959618b9e9289355531d0e0306b9a4, NAME => 'testSplitTableRegionEmptyDaughter,,1676174552598.5a959618b9e9289355531d0e0306b9a4.', STARTKEY => '', ENDKEY => ''}, {ENCODED => e7e2ef82c4b8ad18ce86c0da27ce9ce4, NAME => 'testSplitTableRegionEmptyDaughter,,1676174554196.e7e2ef82c4b8ad18ce86c0da27ce9ce4.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => fd173b0b8daf9a032ee7db30a34f3aae, NAME => 'testSplitTableRegionEmptyDaughter,71,1676174554196.fd173b0b8daf9a032ee7db30a34f3aae.', STARTKEY => '71', ENDKEY => ''}] 2023-02-12 04:02:35,425 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-12 04:02:35,425 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676174555425"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:35,427 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-12 04:02:35,428 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-12 04:02:35,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 44 msec 2023-02-12 04:02:35,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-12 04:02:35,492 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-12 04:02:35,511 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375 (was 376), OpenFileDescriptor=821 (was 823), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=120 (was 122), ProcessCount=163 (was 163), AvailableMemoryMB=4855 (was 4857) 2023-02-12 04:02:35,522 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375, OpenFileDescriptor=821, MaxFileDescriptor=60000, SystemLoadAverage=120, ProcessCount=163, AvailableMemoryMB=4854 2023-02-12 04:02:35,523 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:02:35,525 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-12 04:02:35,526 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-02-12 04:02:35,527 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-12 04:02:35,528 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 04:02:35,529 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 04:02:35,530 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:35,530 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9 empty. 2023-02-12 04:02:35,531 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:35,531 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-12 04:02:35,541 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-12 04:02:35,542 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => fee336dc1d01196d75f25f574e9cc8c9, NAME => 'testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp 2023-02-12 04:02:35,553 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:35,553 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing fee336dc1d01196d75f25f574e9cc8c9, disabling compactions & flushes 2023-02-12 04:02:35,554 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:35,554 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:35,554 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. after waiting 0 ms 2023-02-12 04:02:35,554 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:35,554 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:35,554 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for fee336dc1d01196d75f25f574e9cc8c9: Waiting for close lock at 1676174555553Disabling compacts and flushes for region at 1676174555553Disabling writes for close at 1676174555554 (+1 ms)Writing region close event to WAL at 1676174555554Closed at 1676174555554 2023-02-12 04:02:35,555 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 04:02:35,556 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174555556"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174555556"}]},"ts":"1676174555556"} 2023-02-12 04:02:35,557 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 04:02:35,558 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 04:02:35,558 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174555558"}]},"ts":"1676174555558"} 2023-02-12 04:02:35,559 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-12 04:02:35,561 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-02-12 04:02:35,562 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-12 04:02:35,562 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-12 04:02:35,562 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-12 04:02:35,562 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-12 04:02:35,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=fee336dc1d01196d75f25f574e9cc8c9, ASSIGN}] 2023-02-12 04:02:35,563 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=fee336dc1d01196d75f25f574e9cc8c9, ASSIGN 2023-02-12 04:02:35,563 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=fee336dc1d01196d75f25f574e9cc8c9, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,43977,1676174449184; forceNewPlan=false, retain=false 2023-02-12 04:02:35,714 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-12 04:02:35,716 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=fee336dc1d01196d75f25f574e9cc8c9, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:35,717 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174555716"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174555716"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174555716"}]},"ts":"1676174555716"} 2023-02-12 04:02:35,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure fee336dc1d01196d75f25f574e9cc8c9, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:02:35,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:35,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => fee336dc1d01196d75f25f574e9cc8c9, NAME => 'testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9.', STARTKEY => '', ENDKEY => ''} 2023-02-12 04:02:35,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:02:35,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:35,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:35,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:35,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:35,891 INFO [StoreOpener-fee336dc1d01196d75f25f574e9cc8c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:35,893 DEBUG [StoreOpener-fee336dc1d01196d75f25f574e9cc8c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1 2023-02-12 04:02:35,894 DEBUG [StoreOpener-fee336dc1d01196d75f25f574e9cc8c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1 2023-02-12 04:02:35,894 INFO [StoreOpener-fee336dc1d01196d75f25f574e9cc8c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fee336dc1d01196d75f25f574e9cc8c9 columnFamilyName cf1 2023-02-12 04:02:35,895 INFO [StoreOpener-fee336dc1d01196d75f25f574e9cc8c9-1] regionserver.HStore(310): Store=fee336dc1d01196d75f25f574e9cc8c9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:35,895 INFO [StoreOpener-fee336dc1d01196d75f25f574e9cc8c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:35,896 DEBUG [StoreOpener-fee336dc1d01196d75f25f574e9cc8c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2 2023-02-12 04:02:35,896 DEBUG [StoreOpener-fee336dc1d01196d75f25f574e9cc8c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2 2023-02-12 04:02:35,896 INFO [StoreOpener-fee336dc1d01196d75f25f574e9cc8c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fee336dc1d01196d75f25f574e9cc8c9 columnFamilyName cf2 2023-02-12 04:02:35,897 INFO [StoreOpener-fee336dc1d01196d75f25f574e9cc8c9-1] regionserver.HStore(310): Store=fee336dc1d01196d75f25f574e9cc8c9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:35,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:35,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:35,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:02:35,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:35,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 04:02:35,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened fee336dc1d01196d75f25f574e9cc8c9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66226651, jitterRate=-0.013145998120307922}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:02:35,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for fee336dc1d01196d75f25f574e9cc8c9: Running coprocessor pre-open hook at 1676174555888Writing region info on filesystem at 1676174555888Initializing all the Stores at 1676174555890 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174555890Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174555890Cleaning up temporary data from old regions at 1676174555898 (+8 ms)Cleaning up detritus from prior splits at 1676174555899 (+1 ms)Running coprocessor post-open hooks at 1676174555902 (+3 ms)Region opened successfully at 1676174555902 2023-02-12 04:02:35,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9., pid=143, masterSystemTime=1676174555874 2023-02-12 04:02:35,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:35,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:35,904 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=fee336dc1d01196d75f25f574e9cc8c9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:35,904 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174555904"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174555904"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174555904"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174555904"}]},"ts":"1676174555904"} 2023-02-12 04:02:35,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-12 04:02:35,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure fee336dc1d01196d75f25f574e9cc8c9, server=jenkins-hbase20.apache.org,43977,1676174449184 in 186 msec 2023-02-12 04:02:35,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-12 04:02:35,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=fee336dc1d01196d75f25f574e9cc8c9, ASSIGN in 345 msec 2023-02-12 04:02:35,910 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 04:02:35,910 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174555910"}]},"ts":"1676174555910"} 2023-02-12 04:02:35,911 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-12 04:02:35,913 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 04:02:35,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 387 msec 2023-02-12 04:02:36,043 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:36,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-12 04:02:36,045 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-12 04:02:36,045 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,045 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-12 04:02:36,045 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-12 04:02:36,046 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,046 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,047 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,047 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,047 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,047 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,048 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,048 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:36,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,048 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,048 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,048 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,049 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,049 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,049 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,049 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,049 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,049 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,049 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,049 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,049 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,049 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,049 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,049 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,050 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,050 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,050 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,050 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,050 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,050 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,050 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,050 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,051 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:36,051 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:36,051 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:36,051 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,051 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,051 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. started... 2023-02-12 04:02:36,052 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fee336dc1d01196d75f25f574e9cc8c9 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-12 04:02:36,052 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,053 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,053 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,064 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/12fd9789f38f466b96f4851bb6eca562 2023-02-12 04:02:36,080 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/0121672cac814d30a1679b078a41eaee 2023-02-12 04:02:36,084 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/12fd9789f38f466b96f4851bb6eca562 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/12fd9789f38f466b96f4851bb6eca562 2023-02-12 04:02:36,087 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/12fd9789f38f466b96f4851bb6eca562, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:02:36,088 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/0121672cac814d30a1679b078a41eaee as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/0121672cac814d30a1679b078a41eaee 2023-02-12 04:02:36,090 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/0121672cac814d30a1679b078a41eaee, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:02:36,091 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for fee336dc1d01196d75f25f574e9cc8c9 in 39ms, sequenceid=6, compaction requested=false 2023-02-12 04:02:36,091 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-12 04:02:36,091 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fee336dc1d01196d75f25f574e9cc8c9: 2023-02-12 04:02:36,091 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:36,091 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:36,091 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:36,091 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,091 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,091 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,092 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,092 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,093 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,093 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-12 04:02:36,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b1f5074[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:36,095 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-12 04:02:36,095 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,096 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,096 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,096 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,097 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,097 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:36,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,097 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,097 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,097 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,097 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,097 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:36,097 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:36,097 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,097 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,097 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:36,098 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,098 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,098 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,098 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,098 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,098 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,098 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,098 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,098 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,098 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:02:36,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-12 04:02:36,101 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-12 04:02:36,101 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:36,101 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:36,101 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-12 04:02:36,101 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:36,103 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,103 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,103 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,103 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,103 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,103 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,103 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,103 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,104 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,105 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,107 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,107 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,108 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,110 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,110 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,110 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,110 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,110 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,111 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-12 04:02:36,111 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:36,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,111 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,111 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,111 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:36,111 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:36,111 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,110 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,110 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,111 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,111 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,112 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,112 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,112 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,112 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,112 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,112 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,211 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:36,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:36,235 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:36,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:36,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-12 04:02:36,237 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-12 04:02:36,237 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,237 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-12 04:02:36,237 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-12 04:02:36,238 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,238 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,239 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,239 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,239 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,239 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,239 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,239 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:36,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,240 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,240 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,241 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,241 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,241 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,241 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,241 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,241 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,242 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,242 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,242 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,242 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,242 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:36,242 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:36,242 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,242 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,242 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:36,242 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,242 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,242 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,243 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,242 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. started... 2023-02-12 04:02:36,243 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,243 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,242 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,243 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,243 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fee336dc1d01196d75f25f574e9cc8c9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:36,243 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,243 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,243 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,246 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,246 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,246 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,246 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,246 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,247 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,257 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/4a6baaa4518c4e87a6a0578bb001beef 2023-02-12 04:02:36,268 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/f668f1b9390f4659933d5785a00100dc 2023-02-12 04:02:36,271 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/4a6baaa4518c4e87a6a0578bb001beef as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/4a6baaa4518c4e87a6a0578bb001beef 2023-02-12 04:02:36,275 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/4a6baaa4518c4e87a6a0578bb001beef, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:02:36,276 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/f668f1b9390f4659933d5785a00100dc as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/f668f1b9390f4659933d5785a00100dc 2023-02-12 04:02:36,279 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/f668f1b9390f4659933d5785a00100dc, entries=10, sequenceid=19, filesize=5.0 K 2023-02-12 04:02:36,280 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fee336dc1d01196d75f25f574e9cc8c9 in 37ms, sequenceid=19, compaction requested=false 2023-02-12 04:02:36,280 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fee336dc1d01196d75f25f574e9cc8c9: 2023-02-12 04:02:36,280 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:36,280 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:36,280 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:36,280 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,280 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,280 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,281 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,281 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,281 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,281 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,283 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-12 04:02:36,283 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d51a2df[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:36,283 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-12 04:02:36,283 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,283 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,283 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,283 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,284 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,284 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,284 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:36,284 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,284 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:36,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,284 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,284 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,284 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,284 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:36,284 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,284 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:36,284 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,284 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,284 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,285 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,285 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,285 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,285 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,285 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,285 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,285 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:02:36,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-12 04:02:36,287 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-12 04:02:36,287 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:36,287 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:36,287 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-12 04:02:36,287 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:36,288 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,288 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,288 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,288 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,288 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,289 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,288 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,289 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,289 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,290 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,291 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,291 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,291 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,293 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,293 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,293 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,293 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,293 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,293 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,293 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,293 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,293 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,293 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-12 04:02:36,294 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:36,293 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,293 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,293 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,294 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:36,294 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,294 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:36,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,294 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,294 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,294 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,294 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,294 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,394 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:36,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:36,417 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:36,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:36,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-12 04:02:36,419 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-12 04:02:36,419 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,419 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-12 04:02:36,419 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-12 04:02:36,420 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,420 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,421 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,421 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,421 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,421 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,422 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,422 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,423 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,424 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:36,424 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,424 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,424 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. started... 2023-02-12 04:02:36,424 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fee336dc1d01196d75f25f574e9cc8c9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:36,425 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,425 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,425 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,425 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,425 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,425 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,433 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/f2d7f701b8694c8e91bcbed1285e952e 2023-02-12 04:02:36,445 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/4392bd58c1f745eca09c9ee85a2a2e0e 2023-02-12 04:02:36,450 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/f2d7f701b8694c8e91bcbed1285e952e as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/f2d7f701b8694c8e91bcbed1285e952e 2023-02-12 04:02:36,455 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/f2d7f701b8694c8e91bcbed1285e952e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:02:36,456 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/4392bd58c1f745eca09c9ee85a2a2e0e as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/4392bd58c1f745eca09c9ee85a2a2e0e 2023-02-12 04:02:36,460 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/4392bd58c1f745eca09c9ee85a2a2e0e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-12 04:02:36,461 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fee336dc1d01196d75f25f574e9cc8c9 in 37ms, sequenceid=32, compaction requested=true 2023-02-12 04:02:36,461 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fee336dc1d01196d75f25f574e9cc8c9: 2023-02-12 04:02:36,461 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:36,461 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:36,462 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:36,462 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,462 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,462 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,463 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,463 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,464 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,464 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-12 04:02:36,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7497d22c[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:36,467 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-12 04:02:36,467 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,468 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,468 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,468 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,468 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,468 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,469 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:36,469 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,469 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,469 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,469 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:36,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,469 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:36,469 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,469 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,469 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:36,470 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,470 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,470 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,470 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,471 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,471 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,471 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,471 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,471 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,471 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:02:36,476 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-12 04:02:36,476 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-12 04:02:36,476 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:36,476 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:36,476 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-12 04:02:36,476 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:36,484 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,484 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,484 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,484 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,484 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,484 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,485 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,485 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,485 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,486 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,488 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,489 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,489 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,493 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,493 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,493 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,493 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,493 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,493 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-12 04:02:36,493 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:36,493 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,493 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,493 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,493 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,494 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:36,494 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:36,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,493 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,494 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,493 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,494 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,495 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,495 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,495 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,495 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,495 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,594 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:36,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:36,619 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:36,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:36,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-12 04:02:36,622 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-12 04:02:36,622 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,622 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-12 04:02:36,622 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-12 04:02:36,623 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,623 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,624 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,624 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,624 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,624 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,625 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,625 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:36,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,626 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,626 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,626 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,626 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,627 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,627 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,627 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,627 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,627 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,627 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,627 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,627 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,627 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,627 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,627 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,628 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,628 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,628 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,628 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,628 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,628 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,628 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,628 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,628 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,628 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,628 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:36,628 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:36,628 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:36,629 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. started... 2023-02-12 04:02:36,629 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fee336dc1d01196d75f25f574e9cc8c9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:36,635 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/c20b58d2741c47b8b11689287d11f066 2023-02-12 04:02:36,638 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,638 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,638 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,638 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,638 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,638 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,644 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/a0d7a1df8b7746d79ddcaaef94a490fa 2023-02-12 04:02:36,648 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/c20b58d2741c47b8b11689287d11f066 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c20b58d2741c47b8b11689287d11f066 2023-02-12 04:02:36,651 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c20b58d2741c47b8b11689287d11f066, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:02:36,652 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/a0d7a1df8b7746d79ddcaaef94a490fa as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/a0d7a1df8b7746d79ddcaaef94a490fa 2023-02-12 04:02:36,656 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/a0d7a1df8b7746d79ddcaaef94a490fa, entries=10, sequenceid=45, filesize=5.0 K 2023-02-12 04:02:36,656 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fee336dc1d01196d75f25f574e9cc8c9 in 27ms, sequenceid=45, compaction requested=true 2023-02-12 04:02:36,656 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fee336dc1d01196d75f25f574e9cc8c9: 2023-02-12 04:02:36,656 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:36,657 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:36,657 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:36,657 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,657 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,657 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,658 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,658 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,658 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,658 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-12 04:02:36,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@364e8066[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:36,661 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-12 04:02:36,661 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,662 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,662 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,662 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,662 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:36,662 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,662 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,662 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,662 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:36,662 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,663 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,663 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:36,663 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,663 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,663 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:36,663 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,663 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,664 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,664 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,664 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,664 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,664 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,665 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,665 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,665 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:02:36,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-12 04:02:36,670 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-12 04:02:36,670 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:36,670 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:36,670 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-12 04:02:36,670 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:36,680 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,680 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,680 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,680 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,680 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,680 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,680 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,680 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,681 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,681 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,683 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,683 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,683 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,684 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,684 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,684 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,685 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,685 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-12 04:02:36,685 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:36,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,685 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,685 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,685 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,685 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,685 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,685 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:36,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,685 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,685 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:36,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,685 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,685 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,685 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,686 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,686 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,686 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,686 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,686 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,785 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:36,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:36,806 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:36,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:36,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-12 04:02:36,808 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-12 04:02:36,808 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,808 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-12 04:02:36,808 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-12 04:02:36,809 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,809 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,810 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,810 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,810 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,810 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,810 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,810 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:36,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,811 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,811 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,811 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,812 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,811 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,812 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,812 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,812 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,812 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,812 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,812 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,812 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,812 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,812 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,812 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,812 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,812 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,812 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,812 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,812 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,812 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,813 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,813 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,812 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,813 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,813 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:36,813 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:36,813 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:36,813 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. started... 2023-02-12 04:02:36,813 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fee336dc1d01196d75f25f574e9cc8c9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:36,814 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,814 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,814 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,814 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,814 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,814 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,824 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa 2023-02-12 04:02:36,839 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/b1960c8214fb4c5c9e9d15ca728a8e39 2023-02-12 04:02:36,844 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa 2023-02-12 04:02:36,848 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:02:36,848 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/b1960c8214fb4c5c9e9d15ca728a8e39 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/b1960c8214fb4c5c9e9d15ca728a8e39 2023-02-12 04:02:36,851 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/b1960c8214fb4c5c9e9d15ca728a8e39, entries=10, sequenceid=58, filesize=5.0 K 2023-02-12 04:02:36,851 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fee336dc1d01196d75f25f574e9cc8c9 in 38ms, sequenceid=58, compaction requested=true 2023-02-12 04:02:36,851 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fee336dc1d01196d75f25f574e9cc8c9: 2023-02-12 04:02:36,851 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:36,851 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:36,852 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:36,852 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,852 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,852 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,853 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,853 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,853 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,853 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,855 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-12 04:02:36,855 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27c3ea2b[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:36,855 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-12 04:02:36,855 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,856 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,856 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,856 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,856 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,856 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,856 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:36,856 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,856 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:36,856 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,856 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,857 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:36,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,857 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:36,857 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:36,857 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:36,857 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,857 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,857 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,857 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,857 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,857 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,857 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,858 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:36,858 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,858 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:36,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:02:36,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-12 04:02:36,861 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-12 04:02:36,862 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:36,862 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:36,862 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-12 04:02:36,862 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:36,863 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,863 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:36,863 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,863 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:36,863 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,863 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,863 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,863 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:36,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,864 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,864 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:36,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,866 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:36,867 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:36,867 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,871 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,871 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,871 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,871 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,871 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,871 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,871 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:36,872 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,872 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,872 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-12 04:02:36,872 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:36,872 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:36,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,872 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:36,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,872 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:36,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,872 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:36,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:36,873 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,873 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:36,873 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,873 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:36,873 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,873 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,873 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,896 ERROR [BP-664456210-148.251.75.209-1676174445046 heartbeating to localhost.localdomain/127.0.0.1:37867] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-664456210-148.251.75.209-1676174445046 (Datanode Uuid 484548f8-2ed9-47b7-81ed-a4821ba20939) service to localhost.localdomain/127.0.0.1:37867 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-12 04:02:36,973 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:36,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:36,987 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-02-12 04:02:36,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-12 04:02:36,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-12 04:02:36,989 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-12 04:02:36,989 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,989 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-12 04:02:36,989 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-12 04:02:36,989 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,989 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,990 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,990 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,990 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,990 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:36,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:36,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,990 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:36,990 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 04:02:36,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:36,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-12 04:02:36,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-12 04:02:36,991 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,991 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,991 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,991 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-12 04:02:36,992 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,992 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,992 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 04:02:36,992 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,992 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 04:02:36,992 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 04:02:36,992 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-12 04:02:36,992 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:36,992 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,992 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. started... 2023-02-12 04:02:36,992 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:36,992 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,40151,1676174449139' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,992 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,37065,1676174449218' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:36,992 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fee336dc1d01196d75f25f574e9cc8c9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-12 04:02:36,993 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,993 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,994 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,994 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:36,994 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,994 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:36,998 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/67387cc5e26144e6b3d487b76b0fcc2f 2023-02-12 04:02:37,008 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/ac55ee009435499ca668713dfb3c8d40 2023-02-12 04:02:37,013 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/67387cc5e26144e6b3d487b76b0fcc2f as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/67387cc5e26144e6b3d487b76b0fcc2f 2023-02-12 04:02:37,041 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/67387cc5e26144e6b3d487b76b0fcc2f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:02:37,042 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/ac55ee009435499ca668713dfb3c8d40 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/ac55ee009435499ca668713dfb3c8d40 2023-02-12 04:02:37,045 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/ac55ee009435499ca668713dfb3c8d40, entries=10, sequenceid=71, filesize=5.0 K 2023-02-12 04:02:37,046 INFO [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for fee336dc1d01196d75f25f574e9cc8c9 in 54ms, sequenceid=71, compaction requested=true 2023-02-12 04:02:37,046 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fee336dc1d01196d75f25f574e9cc8c9: 2023-02-12 04:02:37,046 DEBUG [rs(jenkins-hbase20.apache.org,43977,1676174449184)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:37,046 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 04:02:37,046 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 04:02:37,046 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,046 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-12 04:02:37,046 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-12 04:02:37,047 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,047 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:37,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:37,048 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,048 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 04:02:37,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:37,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:37,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:37,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:37,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:37,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:37,049 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43977,1676174449184' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-12 04:02:37,049 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4764f356[Count = 0] remaining members to acquire global barrier 2023-02-12 04:02:37,049 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-12 04:02:37,049 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,050 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,050 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,050 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,050 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:37,050 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,050 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 04:02:37,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,050 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:37,050 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:37,050 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,43977,1676174449184' in zk 2023-02-12 04:02:37,050 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:37,050 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-12 04:02:37,050 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-12 04:02:37,050 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,40151,1676174449139' in zk 2023-02-12 04:02:37,050 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,37065,1676174449218' in zk 2023-02-12 04:02:37,051 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,051 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:37,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:37,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:37,051 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:37,051 DEBUG [member: 'jenkins-hbase20.apache.org,43977,1676174449184' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:37,051 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:37,051 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-12 04:02:37,051 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:37,051 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:37,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:37,051 DEBUG [member: 'jenkins-hbase20.apache.org,37065,1676174449218' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:37,051 DEBUG [member: 'jenkins-hbase20.apache.org,40151,1676174449139' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-12 04:02:37,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:37,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:37,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:37,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:37,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:37,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:37,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:37,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:37,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,43977,1676174449184': 2023-02-12 04:02:37,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43977,1676174449184' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-12 04:02:37,054 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-12 04:02:37,054 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 04:02:37,054 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 04:02:37,054 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-12 04:02:37,054 INFO [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 04:02:37,055 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,055 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,055 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:37,055 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,055 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:37,055 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,055 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:37,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,055 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:37,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 04:02:37,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 04:02:37,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:37,055 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:37,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:37,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:37,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 04:02:37,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:37,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:37,056 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:37,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:37,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 04:02:37,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:37,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:37,057 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:37,057 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:37,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 04:02:37,057 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:37,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-12 04:02:37,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:37,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:37,071 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:37,071 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:37,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:37,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:37,071 DEBUG [(jenkins-hbase20.apache.org,41569,1676174448151)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 04:02:37,071 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:37,071 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:37,071 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 04:02:37,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:37,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:37,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:37,071 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:37,071 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-12 04:02:37,072 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 04:02:37,071 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:37,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:37,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 04:02:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 04:02:37,071 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 04:02:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:37,072 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:37,072 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-12 04:02:37,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 04:02:37,072 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 04:02:37,072 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 04:02:37,072 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-12 04:02:37,072 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:37,072 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,072 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:37,072 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,072 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-12 04:02:37,072 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-12 04:02:37,172 DEBUG [Listener at localhost.localdomain/35207] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 04:02:37,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 04:02:37,179 DEBUG [Listener at localhost.localdomain/35207] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,180 DEBUG [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=fee336dc1d01196d75f25f574e9cc8c9, daughterA=f5675d21343ac8eee8e9165b03e58fa2, daughterB=630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:37,181 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=fee336dc1d01196d75f25f574e9cc8c9, daughterA=f5675d21343ac8eee8e9165b03e58fa2, daughterB=630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:37,181 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=fee336dc1d01196d75f25f574e9cc8c9, daughterA=f5675d21343ac8eee8e9165b03e58fa2, daughterB=630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:37,181 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=fee336dc1d01196d75f25f574e9cc8c9, daughterA=f5675d21343ac8eee8e9165b03e58fa2, daughterB=630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:37,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=fee336dc1d01196d75f25f574e9cc8c9, UNASSIGN}] 2023-02-12 04:02:37,183 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=fee336dc1d01196d75f25f574e9cc8c9, UNASSIGN 2023-02-12 04:02:37,183 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=fee336dc1d01196d75f25f574e9cc8c9, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:37,184 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174557183"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174557183"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174557183"}]},"ts":"1676174557183"} 2023-02-12 04:02:37,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure fee336dc1d01196d75f25f574e9cc8c9, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:02:37,336 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,336 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing fee336dc1d01196d75f25f574e9cc8c9, disabling compactions & flushes 2023-02-12 04:02:37,336 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:37,336 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:37,336 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. after waiting 0 ms 2023-02-12 04:02:37,336 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:37,336 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing fee336dc1d01196d75f25f574e9cc8c9 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-12 04:02:37,344 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/5813e47269dd483f89b5cbb3e1b1eca2 2023-02-12 04:02:37,353 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/9d587c556dd94b22b59989d10ade967c 2023-02-12 04:02:37,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf1/5813e47269dd483f89b5cbb3e1b1eca2 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/5813e47269dd483f89b5cbb3e1b1eca2 2023-02-12 04:02:37,359 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/5813e47269dd483f89b5cbb3e1b1eca2, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:02:37,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/.tmp/cf2/9d587c556dd94b22b59989d10ade967c as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/9d587c556dd94b22b59989d10ade967c 2023-02-12 04:02:37,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/9d587c556dd94b22b59989d10ade967c, entries=8, sequenceid=82, filesize=5.0 K 2023-02-12 04:02:37,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for fee336dc1d01196d75f25f574e9cc8c9 in 27ms, sequenceid=82, compaction requested=true 2023-02-12 04:02:37,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-12 04:02:37,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:37,368 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. 2023-02-12 04:02:37,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for fee336dc1d01196d75f25f574e9cc8c9: Waiting for close lock at 1676174557336Running coprocessor pre-close hooks at 1676174557336Disabling compacts and flushes for region at 1676174557336Disabling writes for close at 1676174557336Obtaining lock to block concurrent updates at 1676174557336Preparing flush snapshotting stores in fee336dc1d01196d75f25f574e9cc8c9 at 1676174557336Finished memstore snapshotting testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676174557336Flushing stores of testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. at 1676174557337 (+1 ms)Flushing fee336dc1d01196d75f25f574e9cc8c9/cf1: creating writer at 1676174557338 (+1 ms)Flushing fee336dc1d01196d75f25f574e9cc8c9/cf1: appending metadata at 1676174557339 (+1 ms)Flushing fee336dc1d01196d75f25f574e9cc8c9/cf1: closing flushed file at 1676174557339Flushing fee336dc1d01196d75f25f574e9cc8c9/cf2: creating writer at 1676174557347 (+8 ms)Flushing fee336dc1d01196d75f25f574e9cc8c9/cf2: appending metadata at 1676174557348 (+1 ms)Flushing fee336dc1d01196d75f25f574e9cc8c9/cf2: closing flushed file at 1676174557348Flushing fee336dc1d01196d75f25f574e9cc8c9/cf1: reopening flushed file at 1676174557357 (+9 ms)Flushing fee336dc1d01196d75f25f574e9cc8c9/cf2: reopening flushed file at 1676174557360 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for fee336dc1d01196d75f25f574e9cc8c9 in 27ms, sequenceid=82, compaction requested=true at 1676174557363 (+3 ms)Writing region close event to WAL at 1676174557365 (+2 ms)Running coprocessor post-close hooks at 1676174557368 (+3 ms)Closed at 1676174557368 2023-02-12 04:02:37,369 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,369 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=fee336dc1d01196d75f25f574e9cc8c9, regionState=CLOSED 2023-02-12 04:02:37,369 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174557369"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174557369"}]},"ts":"1676174557369"} 2023-02-12 04:02:37,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-12 04:02:37,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure fee336dc1d01196d75f25f574e9cc8c9, server=jenkins-hbase20.apache.org,43977,1676174449184 in 186 msec 2023-02-12 04:02:37,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-12 04:02:37,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=fee336dc1d01196d75f25f574e9cc8c9, UNASSIGN in 189 msec 2023-02-12 04:02:37,379 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=fee336dc1d01196d75f25f574e9cc8c9, threads=14 2023-02-12 04:02:37,379 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/0121672cac814d30a1679b078a41eaee for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,379 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/4392bd58c1f745eca09c9ee85a2a2e0e for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,379 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/9d587c556dd94b22b59989d10ade967c for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,379 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/a0d7a1df8b7746d79ddcaaef94a490fa for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,379 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/ac55ee009435499ca668713dfb3c8d40 for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,379 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/b1960c8214fb4c5c9e9d15ca728a8e39 for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,379 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/f668f1b9390f4659933d5785a00100dc for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,379 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/12fd9789f38f466b96f4851bb6eca562 for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,380 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/4a6baaa4518c4e87a6a0578bb001beef for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,380 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/5813e47269dd483f89b5cbb3e1b1eca2 for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,380 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/67387cc5e26144e6b3d487b76b0fcc2f for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,380 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c20b58d2741c47b8b11689287d11f066 for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,380 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,380 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/f2d7f701b8694c8e91bcbed1285e952e for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,526 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/b1960c8214fb4c5c9e9d15ca728a8e39 for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,532 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/0121672cac814d30a1679b078a41eaee for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,560 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/67387cc5e26144e6b3d487b76b0fcc2f for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,575 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c20b58d2741c47b8b11689287d11f066 for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,579 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/a0d7a1df8b7746d79ddcaaef94a490fa for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,579 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/f2d7f701b8694c8e91bcbed1285e952e for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,579 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/f668f1b9390f4659933d5785a00100dc for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,579 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/5813e47269dd483f89b5cbb3e1b1eca2 for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,580 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/9d587c556dd94b22b59989d10ade967c for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,587 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/4a6baaa4518c4e87a6a0578bb001beef for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,588 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/ac55ee009435499ca668713dfb3c8d40 for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:37,896 ERROR [BP-664456210-148.251.75.209-1676174445046 heartbeating to localhost.localdomain/127.0.0.1:37867] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-664456210-148.251.75.209-1676174445046 (Datanode Uuid ba999208-985a-462e-b295-ddee73c8b42c) service to localhost.localdomain/127.0.0.1:37867 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-12 04:02:37,989 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:39,098 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/12fd9789f38f466b96f4851bb6eca562 for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:39,113 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/4392bd58c1f745eca09c9ee85a2a2e0e for region: fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:39,113 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region fee336dc1d01196d75f25f574e9cc8c9 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-12 04:02:39,553 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-12 04:02:39,554 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-12 04:02:39,556 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676174559555"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676174559555"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676174559555"}]},"ts":"1676174559555"} 2023-02-12 04:02:39,556 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174559555"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174559555"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174559555"}]},"ts":"1676174559555"} 2023-02-12 04:02:39,556 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174559555"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174559555"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174559555"}]},"ts":"1676174559555"} 2023-02-12 04:02:39,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=f5675d21343ac8eee8e9165b03e58fa2, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=630dc5e3092b0e401434bef8820e3f5b, ASSIGN}] 2023-02-12 04:02:39,559 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=630dc5e3092b0e401434bef8820e3f5b, ASSIGN 2023-02-12 04:02:39,559 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=630dc5e3092b0e401434bef8820e3f5b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,43977,1676174449184; forceNewPlan=false, retain=false 2023-02-12 04:02:39,559 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=f5675d21343ac8eee8e9165b03e58fa2, ASSIGN 2023-02-12 04:02:39,560 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=f5675d21343ac8eee8e9165b03e58fa2, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,43977,1676174449184; forceNewPlan=false, retain=false 2023-02-12 04:02:39,709 INFO [jenkins-hbase20:41569] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-12 04:02:39,711 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=630dc5e3092b0e401434bef8820e3f5b, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:39,711 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174559711"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174559711"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174559711"}]},"ts":"1676174559711"} 2023-02-12 04:02:39,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=148, state=RUNNABLE; OpenRegionProcedure 630dc5e3092b0e401434bef8820e3f5b, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:02:39,714 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=f5675d21343ac8eee8e9165b03e58fa2, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:39,715 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174559714"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174559714"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174559714"}]},"ts":"1676174559714"} 2023-02-12 04:02:39,716 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=147, state=RUNNABLE; OpenRegionProcedure f5675d21343ac8eee8e9165b03e58fa2, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:02:39,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b. 2023-02-12 04:02:39,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 630dc5e3092b0e401434bef8820e3f5b, NAME => 'testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.', STARTKEY => '41', ENDKEY => ''} 2023-02-12 04:02:39,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:02:39,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:39,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:39,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for 630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:39,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for 630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:39,871 INFO [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:39,872 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1 2023-02-12 04:02:39,872 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1 2023-02-12 04:02:39,873 INFO [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 630dc5e3092b0e401434bef8820e3f5b columnFamilyName cf1 2023-02-12 04:02:39,892 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/12fd9789f38f466b96f4851bb6eca562.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/12fd9789f38f466b96f4851bb6eca562-top 2023-02-12 04:02:39,895 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/4a6baaa4518c4e87a6a0578bb001beef.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/4a6baaa4518c4e87a6a0578bb001beef-top 2023-02-12 04:02:39,898 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/5813e47269dd483f89b5cbb3e1b1eca2.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/5813e47269dd483f89b5cbb3e1b1eca2-top 2023-02-12 04:02:39,901 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/67387cc5e26144e6b3d487b76b0fcc2f.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/67387cc5e26144e6b3d487b76b0fcc2f-top 2023-02-12 04:02:39,904 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/c20b58d2741c47b8b11689287d11f066.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c20b58d2741c47b8b11689287d11f066-top 2023-02-12 04:02:39,907 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa-top 2023-02-12 04:02:39,911 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/f2d7f701b8694c8e91bcbed1285e952e.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/f2d7f701b8694c8e91bcbed1285e952e-top 2023-02-12 04:02:39,911 INFO [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(310): Store=630dc5e3092b0e401434bef8820e3f5b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:39,911 INFO [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:39,911 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 04:02:39,912 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2 2023-02-12 04:02:39,912 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2 2023-02-12 04:02:39,912 INFO [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 630dc5e3092b0e401434bef8820e3f5b columnFamilyName cf2 2023-02-12 04:02:39,927 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/0121672cac814d30a1679b078a41eaee.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/0121672cac814d30a1679b078a41eaee-top 2023-02-12 04:02:39,931 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/4392bd58c1f745eca09c9ee85a2a2e0e.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/4392bd58c1f745eca09c9ee85a2a2e0e-top 2023-02-12 04:02:39,934 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/9d587c556dd94b22b59989d10ade967c.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/9d587c556dd94b22b59989d10ade967c-top 2023-02-12 04:02:39,938 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/a0d7a1df8b7746d79ddcaaef94a490fa.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/a0d7a1df8b7746d79ddcaaef94a490fa-top 2023-02-12 04:02:39,946 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/ac55ee009435499ca668713dfb3c8d40.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/ac55ee009435499ca668713dfb3c8d40-top 2023-02-12 04:02:39,951 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/b1960c8214fb4c5c9e9d15ca728a8e39.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/b1960c8214fb4c5c9e9d15ca728a8e39-top 2023-02-12 04:02:39,954 DEBUG [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/f668f1b9390f4659933d5785a00100dc.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/f668f1b9390f4659933d5785a00100dc-top 2023-02-12 04:02:39,954 INFO [StoreOpener-630dc5e3092b0e401434bef8820e3f5b-1] regionserver.HStore(310): Store=630dc5e3092b0e401434bef8820e3f5b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:39,955 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:39,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:39,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:02:39,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for 630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:39,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened 630dc5e3092b0e401434bef8820e3f5b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61513081, jitterRate=-0.08338366448879242}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:02:39,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for 630dc5e3092b0e401434bef8820e3f5b: Running coprocessor pre-open hook at 1676174559870Writing region info on filesystem at 1676174559870Initializing all the Stores at 1676174559871 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174559871Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174559871Cleaning up temporary data from old regions at 1676174559956 (+85 ms)Cleaning up detritus from prior splits at 1676174559957 (+1 ms)Running coprocessor post-open hooks at 1676174559959 (+2 ms)Region opened successfully at 1676174559959 2023-02-12 04:02:39,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b., pid=149, masterSystemTime=1676174559864 2023-02-12 04:02:39,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:39,960 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:02:39,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:39,960 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b. 2023-02-12 04:02:39,960 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1912): 630dc5e3092b0e401434bef8820e3f5b/cf1 is initiating minor compaction (all files) 2023-02-12 04:02:39,960 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:02:39,960 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 630dc5e3092b0e401434bef8820e3f5b/cf1 in testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b. 2023-02-12 04:02:39,960 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b. 2023-02-12 04:02:39,960 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1912): 630dc5e3092b0e401434bef8820e3f5b/cf2 is initiating minor compaction (all files) 2023-02-12 04:02:39,960 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 630dc5e3092b0e401434bef8820e3f5b/cf2 in testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b. 2023-02-12 04:02:39,961 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/12fd9789f38f466b96f4851bb6eca562.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/12fd9789f38f466b96f4851bb6eca562-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/4a6baaa4518c4e87a6a0578bb001beef.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/4a6baaa4518c4e87a6a0578bb001beef-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/f2d7f701b8694c8e91bcbed1285e952e.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/f2d7f701b8694c8e91bcbed1285e952e-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/c20b58d2741c47b8b11689287d11f066.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c20b58d2741c47b8b11689287d11f066-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/67387cc5e26144e6b3d487b76b0fcc2f.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/67387cc5e26144e6b3d487b76b0fcc2f-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/5813e47269dd483f89b5cbb3e1b1eca2.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/5813e47269dd483f89b5cbb3e1b1eca2-top] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/.tmp, totalSize=34.9 K 2023-02-12 04:02:39,961 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/0121672cac814d30a1679b078a41eaee.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/0121672cac814d30a1679b078a41eaee-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/f668f1b9390f4659933d5785a00100dc.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/f668f1b9390f4659933d5785a00100dc-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/4392bd58c1f745eca09c9ee85a2a2e0e.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/4392bd58c1f745eca09c9ee85a2a2e0e-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/a0d7a1df8b7746d79ddcaaef94a490fa.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/a0d7a1df8b7746d79ddcaaef94a490fa-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/b1960c8214fb4c5c9e9d15ca728a8e39.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/b1960c8214fb4c5c9e9d15ca728a8e39-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/ac55ee009435499ca668713dfb3c8d40.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/ac55ee009435499ca668713dfb3c8d40-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/9d587c556dd94b22b59989d10ade967c.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/9d587c556dd94b22b59989d10ade967c-top] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/.tmp, totalSize=34.9 K 2023-02-12 04:02:39,961 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 0121672cac814d30a1679b078a41eaee.fee336dc1d01196d75f25f574e9cc8c9, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676174556032 2023-02-12 04:02:39,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b. 2023-02-12 04:02:39,961 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 12fd9789f38f466b96f4851bb6eca562.fee336dc1d01196d75f25f574e9cc8c9, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676174556032 2023-02-12 04:02:39,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b. 2023-02-12 04:02:39,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2. 2023-02-12 04:02:39,961 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting f668f1b9390f4659933d5785a00100dc.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676174556215 2023-02-12 04:02:39,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f5675d21343ac8eee8e9165b03e58fa2, NAME => 'testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.', STARTKEY => '', ENDKEY => '41'} 2023-02-12 04:02:39,962 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=630dc5e3092b0e401434bef8820e3f5b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:39,962 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 4a6baaa4518c4e87a6a0578bb001beef.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676174556215 2023-02-12 04:02:39,962 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174559962"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174559962"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174559962"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174559962"}]},"ts":"1676174559962"} 2023-02-12 04:02:39,962 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 4392bd58c1f745eca09c9ee85a2a2e0e.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676174556397 2023-02-12 04:02:39,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-12 04:02:39,962 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting f2d7f701b8694c8e91bcbed1285e952e.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676174556397 2023-02-12 04:02:39,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion f5675d21343ac8eee8e9165b03e58fa2 2023-02-12 04:02:39,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 04:02:39,962 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting a0d7a1df8b7746d79ddcaaef94a490fa.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676174556599 2023-02-12 04:02:39,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7890): checking encryption for f5675d21343ac8eee8e9165b03e58fa2 2023-02-12 04:02:39,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7893): checking classloading for f5675d21343ac8eee8e9165b03e58fa2 2023-02-12 04:02:39,963 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting c20b58d2741c47b8b11689287d11f066.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676174556599 2023-02-12 04:02:39,963 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting b1960c8214fb4c5c9e9d15ca728a8e39.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676174556787 2023-02-12 04:02:39,963 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting c2cbcb24c3ba43a99bca0adc53ef5bfa.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676174556787 2023-02-12 04:02:39,963 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting ac55ee009435499ca668713dfb3c8d40.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676174556975 2023-02-12 04:02:39,963 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] compactions.Compactor(207): Compacting 9d587c556dd94b22b59989d10ade967c.fee336dc1d01196d75f25f574e9cc8c9, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676174557174 2023-02-12 04:02:39,963 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 67387cc5e26144e6b3d487b76b0fcc2f.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676174556975 2023-02-12 04:02:39,963 INFO [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f5675d21343ac8eee8e9165b03e58fa2 2023-02-12 04:02:39,964 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 5813e47269dd483f89b5cbb3e1b1eca2.fee336dc1d01196d75f25f574e9cc8c9, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676174557174 2023-02-12 04:02:39,965 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=148 2023-02-12 04:02:39,965 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1 2023-02-12 04:02:39,965 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1 2023-02-12 04:02:39,965 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=148, state=SUCCESS; OpenRegionProcedure 630dc5e3092b0e401434bef8820e3f5b, server=jenkins-hbase20.apache.org,43977,1676174449184 in 251 msec 2023-02-12 04:02:39,966 INFO [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f5675d21343ac8eee8e9165b03e58fa2 columnFamilyName cf1 2023-02-12 04:02:39,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=630dc5e3092b0e401434bef8820e3f5b, ASSIGN in 407 msec 2023-02-12 04:02:39,973 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] throttle.PressureAwareThroughputController(145): 630dc5e3092b0e401434bef8820e3f5b#cf1#compaction#143 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:02:39,973 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] throttle.PressureAwareThroughputController(145): 630dc5e3092b0e401434bef8820e3f5b#cf2#compaction#142 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:02:39,976 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/12fd9789f38f466b96f4851bb6eca562.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/12fd9789f38f466b96f4851bb6eca562-bottom 2023-02-12 04:02:39,979 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/4a6baaa4518c4e87a6a0578bb001beef.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/4a6baaa4518c4e87a6a0578bb001beef-bottom 2023-02-12 04:02:39,981 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/5813e47269dd483f89b5cbb3e1b1eca2.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/5813e47269dd483f89b5cbb3e1b1eca2-bottom 2023-02-12 04:02:39,984 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/67387cc5e26144e6b3d487b76b0fcc2f.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/67387cc5e26144e6b3d487b76b0fcc2f-bottom 2023-02-12 04:02:39,987 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/c20b58d2741c47b8b11689287d11f066.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c20b58d2741c47b8b11689287d11f066-bottom 2023-02-12 04:02:39,996 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa-bottom 2023-02-12 04:02:39,999 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/f2d7f701b8694c8e91bcbed1285e952e.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/f2d7f701b8694c8e91bcbed1285e952e-bottom 2023-02-12 04:02:39,999 INFO [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(310): Store=f5675d21343ac8eee8e9165b03e58fa2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:40,000 INFO [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f5675d21343ac8eee8e9165b03e58fa2 2023-02-12 04:02:40,000 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/.tmp/cf1/01cf576b673a4ec2a830b89c3b07b813 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/01cf576b673a4ec2a830b89c3b07b813 2023-02-12 04:02:40,000 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2 2023-02-12 04:02:40,000 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2 2023-02-12 04:02:40,001 INFO [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f5675d21343ac8eee8e9165b03e58fa2 columnFamilyName cf2 2023-02-12 04:02:40,003 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 630dc5e3092b0e401434bef8820e3f5b/cf1 of 630dc5e3092b0e401434bef8820e3f5b into 01cf576b673a4ec2a830b89c3b07b813(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:02:40,004 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 630dc5e3092b0e401434bef8820e3f5b: 2023-02-12 04:02:40,004 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b., storeName=630dc5e3092b0e401434bef8820e3f5b/cf1, priority=9, startTime=1676174559959; duration=0sec 2023-02-12 04:02:40,004 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:40,011 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/0121672cac814d30a1679b078a41eaee.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/0121672cac814d30a1679b078a41eaee-bottom 2023-02-12 04:02:40,013 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/4392bd58c1f745eca09c9ee85a2a2e0e.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/4392bd58c1f745eca09c9ee85a2a2e0e-bottom 2023-02-12 04:02:40,016 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/9d587c556dd94b22b59989d10ade967c.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/9d587c556dd94b22b59989d10ade967c-bottom 2023-02-12 04:02:40,018 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/a0d7a1df8b7746d79ddcaaef94a490fa.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/a0d7a1df8b7746d79ddcaaef94a490fa-bottom 2023-02-12 04:02:40,021 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/ac55ee009435499ca668713dfb3c8d40.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/ac55ee009435499ca668713dfb3c8d40-bottom 2023-02-12 04:02:40,023 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/b1960c8214fb4c5c9e9d15ca728a8e39.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/b1960c8214fb4c5c9e9d15ca728a8e39-bottom 2023-02-12 04:02:40,026 DEBUG [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/f668f1b9390f4659933d5785a00100dc.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/f668f1b9390f4659933d5785a00100dc-bottom 2023-02-12 04:02:40,026 INFO [StoreOpener-f5675d21343ac8eee8e9165b03e58fa2-1] regionserver.HStore(310): Store=f5675d21343ac8eee8e9165b03e58fa2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 04:02:40,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2 2023-02-12 04:02:40,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2 2023-02-12 04:02:40,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 04:02:40,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1054): writing seq id for f5675d21343ac8eee8e9165b03e58fa2 2023-02-12 04:02:40,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1071): Opened f5675d21343ac8eee8e9165b03e58fa2; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69920519, jitterRate=0.04189692437648773}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 04:02:40,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(964): Region open journal for f5675d21343ac8eee8e9165b03e58fa2: Running coprocessor pre-open hook at 1676174559962Writing region info on filesystem at 1676174559962Initializing all the Stores at 1676174559963 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174559963Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676174559963Cleaning up temporary data from old regions at 1676174560028 (+65 ms)Cleaning up detritus from prior splits at 1676174560029 (+1 ms)Running coprocessor post-open hooks at 1676174560030 (+1 ms)Region opened successfully at 1676174560030 2023-02-12 04:02:40,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2., pid=150, masterSystemTime=1676174559864 2023-02-12 04:02:40,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:02:40,032 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:02:40,032 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:02:40,032 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2. 2023-02-12 04:02:40,032 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1912): f5675d21343ac8eee8e9165b03e58fa2/cf1 is initiating minor compaction (all files) 2023-02-12 04:02:40,032 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f5675d21343ac8eee8e9165b03e58fa2/cf1 in testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2. 2023-02-12 04:02:40,032 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/12fd9789f38f466b96f4851bb6eca562.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/12fd9789f38f466b96f4851bb6eca562-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/4a6baaa4518c4e87a6a0578bb001beef.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/4a6baaa4518c4e87a6a0578bb001beef-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/f2d7f701b8694c8e91bcbed1285e952e.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/f2d7f701b8694c8e91bcbed1285e952e-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/c20b58d2741c47b8b11689287d11f066.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c20b58d2741c47b8b11689287d11f066-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/67387cc5e26144e6b3d487b76b0fcc2f.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/67387cc5e26144e6b3d487b76b0fcc2f-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/5813e47269dd483f89b5cbb3e1b1eca2.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/5813e47269dd483f89b5cbb3e1b1eca2-bottom] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/.tmp, totalSize=34.9 K 2023-02-12 04:02:40,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2. 2023-02-12 04:02:40,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2. 2023-02-12 04:02:40,033 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 12fd9789f38f466b96f4851bb6eca562.fee336dc1d01196d75f25f574e9cc8c9, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174556032 2023-02-12 04:02:40,033 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=f5675d21343ac8eee8e9165b03e58fa2, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:40,033 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174560033"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676174560033"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676174560033"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676174560033"}]},"ts":"1676174560033"} 2023-02-12 04:02:40,033 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 4a6baaa4518c4e87a6a0578bb001beef.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174556215 2023-02-12 04:02:40,034 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting f2d7f701b8694c8e91bcbed1285e952e.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174556397 2023-02-12 04:02:40,034 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting c20b58d2741c47b8b11689287d11f066.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174556599 2023-02-12 04:02:40,034 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting c2cbcb24c3ba43a99bca0adc53ef5bfa.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174556787 2023-02-12 04:02:40,035 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 67387cc5e26144e6b3d487b76b0fcc2f.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174556975 2023-02-12 04:02:40,035 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 5813e47269dd483f89b5cbb3e1b1eca2.fee336dc1d01196d75f25f574e9cc8c9, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676174557174 2023-02-12 04:02:40,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=147 2023-02-12 04:02:40,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=147, state=SUCCESS; OpenRegionProcedure f5675d21343ac8eee8e9165b03e58fa2, server=jenkins-hbase20.apache.org,43977,1676174449184 in 318 msec 2023-02-12 04:02:40,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-02-12 04:02:40,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=f5675d21343ac8eee8e9165b03e58fa2, ASSIGN in 478 msec 2023-02-12 04:02:40,038 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=fee336dc1d01196d75f25f574e9cc8c9, daughterA=f5675d21343ac8eee8e9165b03e58fa2, daughterB=630dc5e3092b0e401434bef8820e3f5b in 2.8580 sec 2023-02-12 04:02:40,043 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] throttle.PressureAwareThroughputController(145): f5675d21343ac8eee8e9165b03e58fa2#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:02:40,056 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/.tmp/cf1/3775ea9b116c4dae9702426756e09468 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/3775ea9b116c4dae9702426756e09468 2023-02-12 04:02:40,060 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f5675d21343ac8eee8e9165b03e58fa2/cf1 of f5675d21343ac8eee8e9165b03e58fa2 into 3775ea9b116c4dae9702426756e09468(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:02:40,060 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f5675d21343ac8eee8e9165b03e58fa2: 2023-02-12 04:02:40,060 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2., storeName=f5675d21343ac8eee8e9165b03e58fa2/cf1, priority=9, startTime=1676174560031; duration=0sec 2023-02-12 04:02:40,060 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 04:02:40,061 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-12 04:02:40,061 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2. 2023-02-12 04:02:40,061 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1912): f5675d21343ac8eee8e9165b03e58fa2/cf2 is initiating minor compaction (all files) 2023-02-12 04:02:40,061 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f5675d21343ac8eee8e9165b03e58fa2/cf2 in testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2. 2023-02-12 04:02:40,061 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/0121672cac814d30a1679b078a41eaee.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/0121672cac814d30a1679b078a41eaee-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/f668f1b9390f4659933d5785a00100dc.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/f668f1b9390f4659933d5785a00100dc-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/4392bd58c1f745eca09c9ee85a2a2e0e.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/4392bd58c1f745eca09c9ee85a2a2e0e-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/a0d7a1df8b7746d79ddcaaef94a490fa.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/a0d7a1df8b7746d79ddcaaef94a490fa-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/b1960c8214fb4c5c9e9d15ca728a8e39.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/b1960c8214fb4c5c9e9d15ca728a8e39-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/ac55ee009435499ca668713dfb3c8d40.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/ac55ee009435499ca668713dfb3c8d40-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/9d587c556dd94b22b59989d10ade967c.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/9d587c556dd94b22b59989d10ade967c-bottom] into tmpdir=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/.tmp, totalSize=34.9 K 2023-02-12 04:02:40,062 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 0121672cac814d30a1679b078a41eaee.fee336dc1d01196d75f25f574e9cc8c9, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676174556032 2023-02-12 04:02:40,062 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting f668f1b9390f4659933d5785a00100dc.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676174556215 2023-02-12 04:02:40,062 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 4392bd58c1f745eca09c9ee85a2a2e0e.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676174556397 2023-02-12 04:02:40,063 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting a0d7a1df8b7746d79ddcaaef94a490fa.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676174556599 2023-02-12 04:02:40,063 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting b1960c8214fb4c5c9e9d15ca728a8e39.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676174556787 2023-02-12 04:02:40,064 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting ac55ee009435499ca668713dfb3c8d40.fee336dc1d01196d75f25f574e9cc8c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676174556975 2023-02-12 04:02:40,064 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] compactions.Compactor(207): Compacting 9d587c556dd94b22b59989d10ade967c.fee336dc1d01196d75f25f574e9cc8c9, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676174557174 2023-02-12 04:02:40,071 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] throttle.PressureAwareThroughputController(145): f5675d21343ac8eee8e9165b03e58fa2#cf2#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 04:02:40,083 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/.tmp/cf2/67b1769ed39a47bf8389016b2094628e as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/67b1769ed39a47bf8389016b2094628e 2023-02-12 04:02:40,087 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f5675d21343ac8eee8e9165b03e58fa2/cf2 of f5675d21343ac8eee8e9165b03e58fa2 into 67b1769ed39a47bf8389016b2094628e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:02:40,087 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f5675d21343ac8eee8e9165b03e58fa2: 2023-02-12 04:02:40,087 INFO [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2., storeName=f5675d21343ac8eee8e9165b03e58fa2/cf2, priority=9, startTime=1676174560032; duration=0sec 2023-02-12 04:02:40,087 DEBUG [RS:1;jenkins-hbase20:43977-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:40,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43977] ipc.CallRunner(144): callId: 1118 service: ClientService methodName: Scan size: 143 connection: 148.251.75.209:52638 deadline: 1676174620182, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9. is not online on jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:40,295 DEBUG [Listener at localhost.localdomain/35207] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-12 04:02:40,303 INFO [Listener at localhost.localdomain/35207] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-12 04:02:40,305 DEBUG [Listener at localhost.localdomain/35207] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-12 04:02:40,325 WARN [Listener at localhost.localdomain/35207] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-12 04:02:40,327 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-12 04:02:40,327 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$11(2405): Client=jenkins//148.251.75.209 disable testSplitTableRegion 2023-02-12 04:02:40,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-12 04:02:40,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-12 04:02:40,330 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174560330"}]},"ts":"1676174560330"} 2023-02-12 04:02:40,331 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-12 04:02:40,332 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-12 04:02:40,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=f5675d21343ac8eee8e9165b03e58fa2, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=630dc5e3092b0e401434bef8820e3f5b, UNASSIGN}] 2023-02-12 04:02:40,333 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=630dc5e3092b0e401434bef8820e3f5b, UNASSIGN 2023-02-12 04:02:40,333 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=630dc5e3092b0e401434bef8820e3f5b, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:40,334 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174560333"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174560333"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174560333"}]},"ts":"1676174560333"} 2023-02-12 04:02:40,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 630dc5e3092b0e401434bef8820e3f5b, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:02:40,336 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=f5675d21343ac8eee8e9165b03e58fa2, UNASSIGN 2023-02-12 04:02:40,336 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=f5675d21343ac8eee8e9165b03e58fa2, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:40,336 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174560336"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676174560336"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676174560336"}]},"ts":"1676174560336"} 2023-02-12 04:02:40,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure f5675d21343ac8eee8e9165b03e58fa2, server=jenkins-hbase20.apache.org,43977,1676174449184}] 2023-02-12 04:02:40,402 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/.tmp/cf2/8f561bee9c974506b277414e8d5c8ccb as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/8f561bee9c974506b277414e8d5c8ccb 2023-02-12 04:02:40,409 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 630dc5e3092b0e401434bef8820e3f5b/cf2 of 630dc5e3092b0e401434bef8820e3f5b into 8f561bee9c974506b277414e8d5c8ccb(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 04:02:40,409 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 630dc5e3092b0e401434bef8820e3f5b: 2023-02-12 04:02:40,409 INFO [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b., storeName=630dc5e3092b0e401434bef8820e3f5b/cf2, priority=9, startTime=1676174559960; duration=0sec 2023-02-12 04:02:40,409 DEBUG [RS:1;jenkins-hbase20:43977-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 04:02:40,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-12 04:02:40,487 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:40,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 630dc5e3092b0e401434bef8820e3f5b, disabling compactions & flushes 2023-02-12 04:02:40,487 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b. 2023-02-12 04:02:40,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b. 2023-02-12 04:02:40,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b. after waiting 0 ms 2023-02-12 04:02:40,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b. 2023-02-12 04:02:40,489 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/12fd9789f38f466b96f4851bb6eca562.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/12fd9789f38f466b96f4851bb6eca562-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/4a6baaa4518c4e87a6a0578bb001beef.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/4a6baaa4518c4e87a6a0578bb001beef-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/f2d7f701b8694c8e91bcbed1285e952e.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/f2d7f701b8694c8e91bcbed1285e952e-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/c20b58d2741c47b8b11689287d11f066.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c20b58d2741c47b8b11689287d11f066-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/67387cc5e26144e6b3d487b76b0fcc2f.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/67387cc5e26144e6b3d487b76b0fcc2f-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/5813e47269dd483f89b5cbb3e1b1eca2.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/5813e47269dd483f89b5cbb3e1b1eca2-top] to archive 2023-02-12 04:02:40,491 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:02:40,496 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/12fd9789f38f466b96f4851bb6eca562.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/12fd9789f38f466b96f4851bb6eca562.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,500 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/4a6baaa4518c4e87a6a0578bb001beef.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/4a6baaa4518c4e87a6a0578bb001beef.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,502 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/f2d7f701b8694c8e91bcbed1285e952e.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/f2d7f701b8694c8e91bcbed1285e952e.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,505 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/c20b58d2741c47b8b11689287d11f066.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/c20b58d2741c47b8b11689287d11f066.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,507 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,508 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/67387cc5e26144e6b3d487b76b0fcc2f.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/67387cc5e26144e6b3d487b76b0fcc2f.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,510 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/5813e47269dd483f89b5cbb3e1b1eca2.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/5813e47269dd483f89b5cbb3e1b1eca2.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,512 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/0121672cac814d30a1679b078a41eaee.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/0121672cac814d30a1679b078a41eaee-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/f668f1b9390f4659933d5785a00100dc.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/f668f1b9390f4659933d5785a00100dc-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/4392bd58c1f745eca09c9ee85a2a2e0e.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/4392bd58c1f745eca09c9ee85a2a2e0e-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/a0d7a1df8b7746d79ddcaaef94a490fa.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/a0d7a1df8b7746d79ddcaaef94a490fa-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/b1960c8214fb4c5c9e9d15ca728a8e39.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/b1960c8214fb4c5c9e9d15ca728a8e39-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/ac55ee009435499ca668713dfb3c8d40.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/ac55ee009435499ca668713dfb3c8d40-top, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/9d587c556dd94b22b59989d10ade967c.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/9d587c556dd94b22b59989d10ade967c-top] to archive 2023-02-12 04:02:40,513 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:02:40,515 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/0121672cac814d30a1679b078a41eaee.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/0121672cac814d30a1679b078a41eaee.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,516 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/f668f1b9390f4659933d5785a00100dc.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/f668f1b9390f4659933d5785a00100dc.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,518 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/4392bd58c1f745eca09c9ee85a2a2e0e.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/4392bd58c1f745eca09c9ee85a2a2e0e.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,520 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/a0d7a1df8b7746d79ddcaaef94a490fa.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/a0d7a1df8b7746d79ddcaaef94a490fa.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,521 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/b1960c8214fb4c5c9e9d15ca728a8e39.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/b1960c8214fb4c5c9e9d15ca728a8e39.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,523 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/ac55ee009435499ca668713dfb3c8d40.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/ac55ee009435499ca668713dfb3c8d40.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,524 DEBUG [StoreCloser-testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/9d587c556dd94b22b59989d10ade967c.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/9d587c556dd94b22b59989d10ade967c.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-12 04:02:40,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:40,531 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b. 2023-02-12 04:02:40,531 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 630dc5e3092b0e401434bef8820e3f5b: Waiting for close lock at 1676174560487Running coprocessor pre-close hooks at 1676174560487Disabling compacts and flushes for region at 1676174560487Disabling writes for close at 1676174560487Writing region close event to WAL at 1676174560526 (+39 ms)Running coprocessor post-close hooks at 1676174560530 (+4 ms)Closed at 1676174560531 (+1 ms) 2023-02-12 04:02:40,533 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:40,533 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close f5675d21343ac8eee8e9165b03e58fa2 2023-02-12 04:02:40,533 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing f5675d21343ac8eee8e9165b03e58fa2, disabling compactions & flushes 2023-02-12 04:02:40,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2. 2023-02-12 04:02:40,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2. 2023-02-12 04:02:40,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2. after waiting 0 ms 2023-02-12 04:02:40,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2. 2023-02-12 04:02:40,534 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=630dc5e3092b0e401434bef8820e3f5b, regionState=CLOSED 2023-02-12 04:02:40,534 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174560534"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174560534"}]},"ts":"1676174560534"} 2023-02-12 04:02:40,536 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/12fd9789f38f466b96f4851bb6eca562.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/12fd9789f38f466b96f4851bb6eca562-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/4a6baaa4518c4e87a6a0578bb001beef.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/4a6baaa4518c4e87a6a0578bb001beef-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/f2d7f701b8694c8e91bcbed1285e952e.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/f2d7f701b8694c8e91bcbed1285e952e-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/c20b58d2741c47b8b11689287d11f066.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c20b58d2741c47b8b11689287d11f066-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/67387cc5e26144e6b3d487b76b0fcc2f.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/67387cc5e26144e6b3d487b76b0fcc2f-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/5813e47269dd483f89b5cbb3e1b1eca2.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/5813e47269dd483f89b5cbb3e1b1eca2-bottom] to archive 2023-02-12 04:02:40,537 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:02:40,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-12 04:02:40,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 630dc5e3092b0e401434bef8820e3f5b, server=jenkins-hbase20.apache.org,43977,1676174449184 in 202 msec 2023-02-12 04:02:40,539 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/12fd9789f38f466b96f4851bb6eca562.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/12fd9789f38f466b96f4851bb6eca562.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=630dc5e3092b0e401434bef8820e3f5b, UNASSIGN in 205 msec 2023-02-12 04:02:40,541 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/4a6baaa4518c4e87a6a0578bb001beef.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/4a6baaa4518c4e87a6a0578bb001beef.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,542 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/f2d7f701b8694c8e91bcbed1285e952e.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/f2d7f701b8694c8e91bcbed1285e952e.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,543 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/c20b58d2741c47b8b11689287d11f066.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/c20b58d2741c47b8b11689287d11f066.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,545 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,546 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/67387cc5e26144e6b3d487b76b0fcc2f.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/67387cc5e26144e6b3d487b76b0fcc2f.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,547 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/5813e47269dd483f89b5cbb3e1b1eca2.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/5813e47269dd483f89b5cbb3e1b1eca2.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,549 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/0121672cac814d30a1679b078a41eaee.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/0121672cac814d30a1679b078a41eaee-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/f668f1b9390f4659933d5785a00100dc.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/f668f1b9390f4659933d5785a00100dc-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/4392bd58c1f745eca09c9ee85a2a2e0e.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/4392bd58c1f745eca09c9ee85a2a2e0e-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/a0d7a1df8b7746d79ddcaaef94a490fa.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/a0d7a1df8b7746d79ddcaaef94a490fa-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/b1960c8214fb4c5c9e9d15ca728a8e39.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/b1960c8214fb4c5c9e9d15ca728a8e39-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/ac55ee009435499ca668713dfb3c8d40.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/ac55ee009435499ca668713dfb3c8d40-bottom, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/9d587c556dd94b22b59989d10ade967c.fee336dc1d01196d75f25f574e9cc8c9->hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/9d587c556dd94b22b59989d10ade967c-bottom] to archive 2023-02-12 04:02:40,550 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 04:02:40,551 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/0121672cac814d30a1679b078a41eaee.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/0121672cac814d30a1679b078a41eaee.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,552 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/f668f1b9390f4659933d5785a00100dc.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/f668f1b9390f4659933d5785a00100dc.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,553 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/4392bd58c1f745eca09c9ee85a2a2e0e.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/4392bd58c1f745eca09c9ee85a2a2e0e.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,554 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/a0d7a1df8b7746d79ddcaaef94a490fa.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/a0d7a1df8b7746d79ddcaaef94a490fa.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,554 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/b1960c8214fb4c5c9e9d15ca728a8e39.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/b1960c8214fb4c5c9e9d15ca728a8e39.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,555 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/ac55ee009435499ca668713dfb3c8d40.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/ac55ee009435499ca668713dfb3c8d40.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,556 DEBUG [StoreCloser-testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/9d587c556dd94b22b59989d10ade967c.fee336dc1d01196d75f25f574e9cc8c9 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/9d587c556dd94b22b59989d10ade967c.fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-12 04:02:40,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:40,559 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2. 2023-02-12 04:02:40,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for f5675d21343ac8eee8e9165b03e58fa2: Waiting for close lock at 1676174560533Running coprocessor pre-close hooks at 1676174560533Disabling compacts and flushes for region at 1676174560533Disabling writes for close at 1676174560534 (+1 ms)Writing region close event to WAL at 1676174560557 (+23 ms)Running coprocessor post-close hooks at 1676174560559 (+2 ms)Closed at 1676174560559 2023-02-12 04:02:40,560 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed f5675d21343ac8eee8e9165b03e58fa2 2023-02-12 04:02:40,561 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=f5675d21343ac8eee8e9165b03e58fa2, regionState=CLOSED 2023-02-12 04:02:40,561 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676174560561"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676174560561"}]},"ts":"1676174560561"} 2023-02-12 04:02:40,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-12 04:02:40,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure f5675d21343ac8eee8e9165b03e58fa2, server=jenkins-hbase20.apache.org,43977,1676174449184 in 225 msec 2023-02-12 04:02:40,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-02-12 04:02:40,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=f5675d21343ac8eee8e9165b03e58fa2, UNASSIGN in 230 msec 2023-02-12 04:02:40,564 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676174560564"}]},"ts":"1676174560564"} 2023-02-12 04:02:40,565 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-12 04:02:40,566 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-12 04:02:40,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 239 msec 2023-02-12 04:02:40,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-12 04:02:40,633 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-12 04:02:40,634 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.HMaster$5(2215): Client=jenkins//148.251.75.209 delete testSplitTableRegion 2023-02-12 04:02:40,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-12 04:02:40,637 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-12 04:02:40,638 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-12 04:02:40,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-12 04:02:40,647 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,647 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:40,647 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2 2023-02-12 04:02:40,649 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/recovered.edits] 2023-02-12 04:02:40,649 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/recovered.edits] 2023-02-12 04:02:40,650 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2, FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/recovered.edits] 2023-02-12 04:02:40,652 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/01cf576b673a4ec2a830b89c3b07b813 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf1/01cf576b673a4ec2a830b89c3b07b813 2023-02-12 04:02:40,653 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/3775ea9b116c4dae9702426756e09468 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf1/3775ea9b116c4dae9702426756e09468 2023-02-12 04:02:40,653 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/12fd9789f38f466b96f4851bb6eca562 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/12fd9789f38f466b96f4851bb6eca562 2023-02-12 04:02:40,655 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/8f561bee9c974506b277414e8d5c8ccb to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/cf2/8f561bee9c974506b277414e8d5c8ccb 2023-02-12 04:02:40,655 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/4a6baaa4518c4e87a6a0578bb001beef to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/4a6baaa4518c4e87a6a0578bb001beef 2023-02-12 04:02:40,655 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/67b1769ed39a47bf8389016b2094628e to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/cf2/67b1769ed39a47bf8389016b2094628e 2023-02-12 04:02:40,656 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/5813e47269dd483f89b5cbb3e1b1eca2 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/5813e47269dd483f89b5cbb3e1b1eca2 2023-02-12 04:02:40,657 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/67387cc5e26144e6b3d487b76b0fcc2f to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/67387cc5e26144e6b3d487b76b0fcc2f 2023-02-12 04:02:40,657 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/recovered.edits/90.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2/recovered.edits/90.seqid 2023-02-12 04:02:40,657 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/recovered.edits/90.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b/recovered.edits/90.seqid 2023-02-12 04:02:40,658 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/630dc5e3092b0e401434bef8820e3f5b 2023-02-12 04:02:40,658 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/f5675d21343ac8eee8e9165b03e58fa2 2023-02-12 04:02:40,658 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c20b58d2741c47b8b11689287d11f066 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c20b58d2741c47b8b11689287d11f066 2023-02-12 04:02:40,659 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/c2cbcb24c3ba43a99bca0adc53ef5bfa 2023-02-12 04:02:40,660 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/f2d7f701b8694c8e91bcbed1285e952e to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf1/f2d7f701b8694c8e91bcbed1285e952e 2023-02-12 04:02:40,662 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/0121672cac814d30a1679b078a41eaee to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/0121672cac814d30a1679b078a41eaee 2023-02-12 04:02:40,664 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/4392bd58c1f745eca09c9ee85a2a2e0e to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/4392bd58c1f745eca09c9ee85a2a2e0e 2023-02-12 04:02:40,665 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/9d587c556dd94b22b59989d10ade967c to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/9d587c556dd94b22b59989d10ade967c 2023-02-12 04:02:40,666 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/a0d7a1df8b7746d79ddcaaef94a490fa to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/a0d7a1df8b7746d79ddcaaef94a490fa 2023-02-12 04:02:40,667 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/ac55ee009435499ca668713dfb3c8d40 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/ac55ee009435499ca668713dfb3c8d40 2023-02-12 04:02:40,668 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/b1960c8214fb4c5c9e9d15ca728a8e39 to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/b1960c8214fb4c5c9e9d15ca728a8e39 2023-02-12 04:02:40,670 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/f668f1b9390f4659933d5785a00100dc to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/cf2/f668f1b9390f4659933d5785a00100dc 2023-02-12 04:02:40,672 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/recovered.edits/85.seqid to hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/archive/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9/recovered.edits/85.seqid 2023-02-12 04:02:40,672 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/.tmp/data/default/testSplitTableRegion/fee336dc1d01196d75f25f574e9cc8c9 2023-02-12 04:02:40,672 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-12 04:02:40,674 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-12 04:02:40,676 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-12 04:02:40,678 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-12 04:02:40,678 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-12 04:02:40,679 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-12 04:02:40,679 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174560679"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:40,679 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174560679"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:40,679 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676174560679"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:40,680 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-12 04:02:40,681 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => fee336dc1d01196d75f25f574e9cc8c9, NAME => 'testSplitTableRegion,,1676174555526.fee336dc1d01196d75f25f574e9cc8c9.', STARTKEY => '', ENDKEY => ''}, {ENCODED => f5675d21343ac8eee8e9165b03e58fa2, NAME => 'testSplitTableRegion,,1676174557179.f5675d21343ac8eee8e9165b03e58fa2.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 630dc5e3092b0e401434bef8820e3f5b, NAME => 'testSplitTableRegion,41,1676174557179.630dc5e3092b0e401434bef8820e3f5b.', STARTKEY => '41', ENDKEY => ''}] 2023-02-12 04:02:40,681 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-12 04:02:40,681 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676174560681"}]},"ts":"9223372036854775807"} 2023-02-12 04:02:40,682 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-12 04:02:40,683 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-12 04:02:40,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 49 msec 2023-02-12 04:02:40,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41569] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-12 04:02:40,742 INFO [Listener at localhost.localdomain/35207] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-12 04:02:40,760 INFO [Listener at localhost.localdomain/35207] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=376 (was 375) - Thread LEAK? -, OpenFileDescriptor=829 (was 821) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=110 (was 120), ProcessCount=163 (was 163), AvailableMemoryMB=4790 (was 4854) 2023-02-12 04:02:40,760 INFO [Listener at localhost.localdomain/35207] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-12 04:02:40,764 INFO [Listener at localhost.localdomain/35207] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 04:02:40,764 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x11de39c7 to 127.0.0.1:56679 2023-02-12 04:02:40,764 DEBUG [Listener at localhost.localdomain/35207] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 04:02:40,765 DEBUG [Listener at localhost.localdomain/35207] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-12 04:02:40,765 DEBUG [Listener at localhost.localdomain/35207] util.JVMClusterUtil(257): Found active master hash=1229709319, stopped=false 2023-02-12 04:02:40,766 INFO [Listener at localhost.localdomain/35207] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase20.apache.org,41569,1676174448151 2023-02-12 04:02:40,893 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-12 04:02:40,893 INFO [Listener at localhost.localdomain/35207] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 04:02:40,893 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-12 04:02:40,893 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-12 04:02:40,893 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-12 04:02:40,894 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 04:02:40,894 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-12 04:02:40,894 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-12 04:02:40,894 DEBUG [Listener at localhost.localdomain/35207] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x42e84285 to 127.0.0.1:56679 2023-02-12 04:02:40,894 DEBUG [Listener at localhost.localdomain/35207] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 04:02:40,894 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-12 04:02:40,894 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-12 04:02:40,894 INFO [Listener at localhost.localdomain/35207] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,40151,1676174449139' ***** 2023-02-12 04:02:40,895 INFO [Listener at localhost.localdomain/35207] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-12 04:02:40,895 INFO [Listener at localhost.localdomain/35207] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,43977,1676174449184' ***** 2023-02-12 04:02:40,895 INFO [Listener at localhost.localdomain/35207] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-12 04:02:40,895 INFO [Listener at localhost.localdomain/35207] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,37065,1676174449218' ***** 2023-02-12 04:02:40,895 INFO [Listener at localhost.localdomain/35207] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-12 04:02:40,895 INFO [RS:0;jenkins-hbase20:40151] regionserver.HeapMemoryManager(220): Stopping 2023-02-12 04:02:40,895 INFO [RS:1;jenkins-hbase20:43977] regionserver.HeapMemoryManager(220): Stopping 2023-02-12 04:02:40,895 INFO [RS:2;jenkins-hbase20:37065] regionserver.HeapMemoryManager(220): Stopping 2023-02-12 04:02:40,895 INFO [RS:1;jenkins-hbase20:43977] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-12 04:02:40,895 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-12 04:02:40,895 INFO [RS:2;jenkins-hbase20:37065] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-12 04:02:40,895 INFO [RS:1;jenkins-hbase20:43977] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-12 04:02:40,895 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-12 04:02:40,895 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-12 04:02:40,895 INFO [RS:0;jenkins-hbase20:40151] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-12 04:02:40,899 INFO [RS:2;jenkins-hbase20:37065] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-12 04:02:40,899 INFO [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(3304): Received CLOSE for 6b2660cb506f26cb2f51dda8314eff58 2023-02-12 04:02:40,899 INFO [RS:0;jenkins-hbase20:40151] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-12 04:02:40,899 INFO [RS:2;jenkins-hbase20:37065] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:40,899 INFO [RS:0;jenkins-hbase20:40151] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:40,899 DEBUG [RS:2;jenkins-hbase20:37065] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4c9b813c to 127.0.0.1:56679 2023-02-12 04:02:40,899 DEBUG [RS:2;jenkins-hbase20:37065] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 04:02:40,899 DEBUG [RS:0;jenkins-hbase20:40151] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4df21505 to 127.0.0.1:56679 2023-02-12 04:02:40,899 INFO [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:40,899 DEBUG [RS:0;jenkins-hbase20:40151] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 04:02:40,899 INFO [RS:2;jenkins-hbase20:37065] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,37065,1676174449218; all regions closed. 2023-02-12 04:02:40,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 6b2660cb506f26cb2f51dda8314eff58, disabling compactions & flushes 2023-02-12 04:02:40,899 INFO [RS:0;jenkins-hbase20:40151] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,40151,1676174449139; all regions closed. 2023-02-12 04:02:40,899 DEBUG [RS:1;jenkins-hbase20:43977] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6a8349c7 to 127.0.0.1:56679 2023-02-12 04:02:40,899 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. 2023-02-12 04:02:40,899 DEBUG [RS:1;jenkins-hbase20:43977] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 04:02:40,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. 2023-02-12 04:02:40,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. after waiting 0 ms 2023-02-12 04:02:40,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. 2023-02-12 04:02:40,900 INFO [RS:1;jenkins-hbase20:43977] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-12 04:02:40,900 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 6b2660cb506f26cb2f51dda8314eff58 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-12 04:02:40,900 INFO [RS:1;jenkins-hbase20:43977] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-12 04:02:40,900 INFO [RS:1;jenkins-hbase20:43977] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-12 04:02:40,900 INFO [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-12 04:02:40,900 INFO [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-12 04:02:40,900 DEBUG [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740, 6b2660cb506f26cb2f51dda8314eff58=hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58.} 2023-02-12 04:02:40,900 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-12 04:02:40,900 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-12 04:02:40,900 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-12 04:02:40,900 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-12 04:02:40,900 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-12 04:02:40,900 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.49 KB heapSize=125.82 KB 2023-02-12 04:02:40,901 DEBUG [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1505): Waiting on 1588230740, 6b2660cb506f26cb2f51dda8314eff58 2023-02-12 04:02:40,913 DEBUG [RS:0;jenkins-hbase20:40151] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/oldWALs 2023-02-12 04:02:40,913 DEBUG [RS:2;jenkins-hbase20:37065] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/oldWALs 2023-02-12 04:02:40,913 INFO [RS:0;jenkins-hbase20:40151] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C40151%2C1676174449139:(num 1676174450636) 2023-02-12 04:02:40,913 INFO [RS:2;jenkins-hbase20:37065] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C37065%2C1676174449218:(num 1676174450636) 2023-02-12 04:02:40,914 DEBUG [RS:2;jenkins-hbase20:37065] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 04:02:40,914 DEBUG [RS:0;jenkins-hbase20:40151] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 04:02:40,914 INFO [RS:2;jenkins-hbase20:37065] regionserver.LeaseManager(133): Closed leases 2023-02-12 04:02:40,914 INFO [RS:0;jenkins-hbase20:40151] regionserver.LeaseManager(133): Closed leases 2023-02-12 04:02:40,914 INFO [RS:0;jenkins-hbase20:40151] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-12 04:02:40,914 INFO [RS:2;jenkins-hbase20:37065] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-12 04:02:40,919 INFO [RS:2;jenkins-hbase20:37065] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-12 04:02:40,919 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-12 04:02:40,919 INFO [RS:2;jenkins-hbase20:37065] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-12 04:02:40,919 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-12 04:02:40,919 INFO [RS:0;jenkins-hbase20:40151] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-12 04:02:40,919 INFO [RS:2;jenkins-hbase20:37065] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-12 04:02:40,919 INFO [RS:0;jenkins-hbase20:40151] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-12 04:02:40,920 INFO [RS:0;jenkins-hbase20:40151] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-12 04:02:40,921 INFO [RS:0;jenkins-hbase20:40151] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:40151 2023-02-12 04:02:40,921 INFO [RS:2;jenkins-hbase20:37065] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:37065 2023-02-12 04:02:40,925 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:40,925 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:40,925 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-12 04:02:40,925 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,40151,1676174449139 2023-02-12 04:02:40,925 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-12 04:02:40,925 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-12 04:02:40,926 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-12 04:02:40,926 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:40,926 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:40,926 ERROR [Listener at localhost.localdomain/35207-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@64947bfe rejected from java.util.concurrent.ThreadPoolExecutor@60abe3b4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 04:02:40,926 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,37065,1676174449218 2023-02-12 04:02:40,926 ERROR [Listener at localhost.localdomain/35207-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@534ad81c rejected from java.util.concurrent.ThreadPoolExecutor@60abe3b4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 04:02:40,926 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,37065,1676174449218] 2023-02-12 04:02:40,927 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,37065,1676174449218; numProcessing=1 2023-02-12 04:02:40,927 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,37065,1676174449218 already deleted, retry=false 2023-02-12 04:02:40,927 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,37065,1676174449218 expired; onlineServers=2 2023-02-12 04:02:40,927 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,40151,1676174449139] 2023-02-12 04:02:40,927 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,40151,1676174449139; numProcessing=2 2023-02-12 04:02:40,928 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,40151,1676174449139 already deleted, retry=false 2023-02-12 04:02:40,928 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,40151,1676174449139 expired; onlineServers=1 2023-02-12 04:02:40,992 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-12 04:02:40,992 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-12 04:02:40,992 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-12 04:02:41,027 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-12 04:02:41,027 INFO [RS:0;jenkins-hbase20:40151] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,40151,1676174449139; zookeeper connection closed. 2023-02-12 04:02:41,027 ERROR [Listener at localhost.localdomain/35207-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2b4fe57f rejected from java.util.concurrent.ThreadPoolExecutor@60abe3b4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 04:02:41,028 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:40151-0x10154d814950001, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-12 04:02:41,028 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@b31811b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@b31811b 2023-02-12 04:02:41,028 ERROR [Listener at localhost.localdomain/35207-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4009341e rejected from java.util.concurrent.ThreadPoolExecutor@60abe3b4[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 04:02:41,101 DEBUG [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1505): Waiting on 1588230740, 6b2660cb506f26cb2f51dda8314eff58 2023-02-12 04:02:41,127 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-12 04:02:41,127 INFO [RS:2;jenkins-hbase20:37065] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,37065,1676174449218; zookeeper connection closed. 2023-02-12 04:02:41,128 ERROR [Listener at localhost.localdomain/35207-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5b2686cb rejected from java.util.concurrent.ThreadPoolExecutor@5f190977[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 04:02:41,128 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@11a0efb2] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@11a0efb2 2023-02-12 04:02:41,128 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:37065-0x10154d814950003, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-12 04:02:41,128 ERROR [Listener at localhost.localdomain/35207-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@443424ee rejected from java.util.concurrent.ThreadPoolExecutor@5f190977[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 04:02:41,302 DEBUG [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1505): Waiting on 1588230740, 6b2660cb506f26cb2f51dda8314eff58 2023-02-12 04:02:41,324 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/namespace/6b2660cb506f26cb2f51dda8314eff58/.tmp/info/4ff0c1495b214ec183fce965fe748048 2023-02-12 04:02:41,328 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.37 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/.tmp/info/7454171843ed4e35a2f7217efe2b737c 2023-02-12 04:02:41,337 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7454171843ed4e35a2f7217efe2b737c 2023-02-12 04:02:41,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/namespace/6b2660cb506f26cb2f51dda8314eff58/.tmp/info/4ff0c1495b214ec183fce965fe748048 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/namespace/6b2660cb506f26cb2f51dda8314eff58/info/4ff0c1495b214ec183fce965fe748048 2023-02-12 04:02:41,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/namespace/6b2660cb506f26cb2f51dda8314eff58/info/4ff0c1495b214ec183fce965fe748048, entries=2, sequenceid=6, filesize=4.8 K 2023-02-12 04:02:41,343 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 6b2660cb506f26cb2f51dda8314eff58 in 443ms, sequenceid=6, compaction requested=false 2023-02-12 04:02:41,354 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/.tmp/rep_barrier/bee94621bd234e55a95b37cdf29c3b1b 2023-02-12 04:02:41,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/namespace/6b2660cb506f26cb2f51dda8314eff58/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-12 04:02:41,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:41,355 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. 2023-02-12 04:02:41,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 6b2660cb506f26cb2f51dda8314eff58: Waiting for close lock at 1676174560899Running coprocessor pre-close hooks at 1676174560899Disabling compacts and flushes for region at 1676174560899Disabling writes for close at 1676174560900 (+1 ms)Obtaining lock to block concurrent updates at 1676174560900Preparing flush snapshotting stores in 6b2660cb506f26cb2f51dda8314eff58 at 1676174560900Finished memstore snapshotting hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676174560900Flushing stores of hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. at 1676174560901 (+1 ms)Flushing 6b2660cb506f26cb2f51dda8314eff58/info: creating writer at 1676174560901Flushing 6b2660cb506f26cb2f51dda8314eff58/info: appending metadata at 1676174560907 (+6 ms)Flushing 6b2660cb506f26cb2f51dda8314eff58/info: closing flushed file at 1676174560907Flushing 6b2660cb506f26cb2f51dda8314eff58/info: reopening flushed file at 1676174561338 (+431 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 6b2660cb506f26cb2f51dda8314eff58 in 443ms, sequenceid=6, compaction requested=false at 1676174561343 (+5 ms)Writing region close event to WAL at 1676174561351 (+8 ms)Running coprocessor post-close hooks at 1676174561354 (+3 ms)Closed at 1676174561355 (+1 ms) 2023-02-12 04:02:41,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676174451158.6b2660cb506f26cb2f51dda8314eff58. 2023-02-12 04:02:41,357 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bee94621bd234e55a95b37cdf29c3b1b 2023-02-12 04:02:41,369 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/.tmp/table/03240830bdc94f99b4db7c14cc5a3a49 2023-02-12 04:02:41,371 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03240830bdc94f99b4db7c14cc5a3a49 2023-02-12 04:02:41,372 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/.tmp/info/7454171843ed4e35a2f7217efe2b737c as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/info/7454171843ed4e35a2f7217efe2b737c 2023-02-12 04:02:41,374 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7454171843ed4e35a2f7217efe2b737c 2023-02-12 04:02:41,374 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/info/7454171843ed4e35a2f7217efe2b737c, entries=59, sequenceid=210, filesize=11.1 K 2023-02-12 04:02:41,375 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/.tmp/rep_barrier/bee94621bd234e55a95b37cdf29c3b1b as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/rep_barrier/bee94621bd234e55a95b37cdf29c3b1b 2023-02-12 04:02:41,377 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bee94621bd234e55a95b37cdf29c3b1b 2023-02-12 04:02:41,377 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/rep_barrier/bee94621bd234e55a95b37cdf29c3b1b, entries=24, sequenceid=210, filesize=7.5 K 2023-02-12 04:02:41,377 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/.tmp/table/03240830bdc94f99b4db7c14cc5a3a49 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/table/03240830bdc94f99b4db7c14cc5a3a49 2023-02-12 04:02:41,379 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03240830bdc94f99b4db7c14cc5a3a49 2023-02-12 04:02:41,380 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/table/03240830bdc94f99b4db7c14cc5a3a49, entries=36, sequenceid=210, filesize=8.0 K 2023-02-12 04:02:41,380 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 480ms, sequenceid=210, compaction requested=false 2023-02-12 04:02:41,384 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-12 04:02:41,385 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-12 04:02:41,385 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-12 04:02:41,386 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-12 04:02:41,386 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676174560900Running coprocessor pre-close hooks at 1676174560900Disabling compacts and flushes for region at 1676174560900Disabling writes for close at 1676174560900Obtaining lock to block concurrent updates at 1676174560901 (+1 ms)Preparing flush snapshotting stores in 1588230740 at 1676174560901Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81396, getHeapSize=128792, getOffHeapSize=0, getCellsCount=617 at 1676174560901Flushing stores of hbase:meta,,1.1588230740 at 1676174560901Flushing 1588230740/info: creating writer at 1676174560901Flushing 1588230740/info: appending metadata at 1676174560914 (+13 ms)Flushing 1588230740/info: closing flushed file at 1676174560914Flushing 1588230740/rep_barrier: creating writer at 1676174561338 (+424 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676174561340 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676174561340Flushing 1588230740/table: creating writer at 1676174561357 (+17 ms)Flushing 1588230740/table: appending metadata at 1676174561359 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676174561359Flushing 1588230740/info: reopening flushed file at 1676174561372 (+13 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676174561375 (+3 ms)Flushing 1588230740/table: reopening flushed file at 1676174561377 (+2 ms)Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 480ms, sequenceid=210, compaction requested=false at 1676174561380 (+3 ms)Writing region close event to WAL at 1676174561382 (+2 ms)Running coprocessor post-close hooks at 1676174561385 (+3 ms)Closed at 1676174561386 (+1 ms) 2023-02-12 04:02:41,386 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-12 04:02:41,475 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-12 04:02:41,475 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-12 04:02:41,502 INFO [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,43977,1676174449184; all regions closed. 2023-02-12 04:02:41,513 DEBUG [RS:1;jenkins-hbase20:43977] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/oldWALs 2023-02-12 04:02:41,513 INFO [RS:1;jenkins-hbase20:43977] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C43977%2C1676174449184.meta:.meta(num 1676174450887) 2023-02-12 04:02:41,519 DEBUG [RS:1;jenkins-hbase20:43977] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/oldWALs 2023-02-12 04:02:41,519 INFO [RS:1;jenkins-hbase20:43977] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C43977%2C1676174449184:(num 1676174450636) 2023-02-12 04:02:41,519 DEBUG [RS:1;jenkins-hbase20:43977] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 04:02:41,519 INFO [RS:1;jenkins-hbase20:43977] regionserver.LeaseManager(133): Closed leases 2023-02-12 04:02:41,520 INFO [RS:1;jenkins-hbase20:43977] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-12 04:02:41,520 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-12 04:02:41,521 INFO [RS:1;jenkins-hbase20:43977] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:43977 2023-02-12 04:02:41,522 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,43977,1676174449184 2023-02-12 04:02:41,522 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-12 04:02:41,523 ERROR [Listener at localhost.localdomain/35207-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7f680263 rejected from java.util.concurrent.ThreadPoolExecutor@7b579956[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 04:02:41,523 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,43977,1676174449184] 2023-02-12 04:02:41,523 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,43977,1676174449184; numProcessing=3 2023-02-12 04:02:41,524 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,43977,1676174449184 already deleted, retry=false 2023-02-12 04:02:41,524 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,43977,1676174449184 expired; onlineServers=0 2023-02-12 04:02:41,524 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,41569,1676174448151' ***** 2023-02-12 04:02:41,525 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-12 04:02:41,526 DEBUG [M:0;jenkins-hbase20:41569] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@876ea3c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-02-12 04:02:41,526 INFO [M:0;jenkins-hbase20:41569] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,41569,1676174448151 2023-02-12 04:02:41,526 INFO [M:0;jenkins-hbase20:41569] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,41569,1676174448151; all regions closed. 2023-02-12 04:02:41,526 DEBUG [M:0;jenkins-hbase20:41569] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 04:02:41,526 DEBUG [M:0;jenkins-hbase20:41569] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-12 04:02:41,526 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-12 04:02:41,526 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676174450294] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1676174450294,5,FailOnTimeoutGroup] 2023-02-12 04:02:41,526 DEBUG [M:0;jenkins-hbase20:41569] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-12 04:02:41,526 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676174450294] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1676174450294,5,FailOnTimeoutGroup] 2023-02-12 04:02:41,527 INFO [M:0;jenkins-hbase20:41569] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-12 04:02:41,527 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-12 04:02:41,527 INFO [M:0;jenkins-hbase20:41569] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-12 04:02:41,527 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 04:02:41,527 INFO [M:0;jenkins-hbase20:41569] hbase.ChoreService(369): Chore service for: master/jenkins-hbase20:0 had [] on shutdown 2023-02-12 04:02:41,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-12 04:02:41,527 DEBUG [M:0;jenkins-hbase20:41569] master.HMaster(1502): Stopping service threads 2023-02-12 04:02:41,527 INFO [M:0;jenkins-hbase20:41569] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-12 04:02:41,527 ERROR [M:0;jenkins-hbase20:41569] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-12 04:02:41,527 INFO [M:0;jenkins-hbase20:41569] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 04:02:41,528 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-12 04:02:41,528 DEBUG [M:0;jenkins-hbase20:41569] zookeeper.ZKUtil(398): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-12 04:02:41,528 WARN [M:0;jenkins-hbase20:41569] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-12 04:02:41,528 INFO [M:0;jenkins-hbase20:41569] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 04:02:41,528 INFO [M:0;jenkins-hbase20:41569] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-12 04:02:41,528 DEBUG [M:0;jenkins-hbase20:41569] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-12 04:02:41,528 INFO [M:0;jenkins-hbase20:41569] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 04:02:41,528 DEBUG [M:0;jenkins-hbase20:41569] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 04:02:41,528 DEBUG [M:0;jenkins-hbase20:41569] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-12 04:02:41,528 DEBUG [M:0;jenkins-hbase20:41569] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 04:02:41,528 INFO [M:0;jenkins-hbase20:41569] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.56 KB heapSize=719.54 KB 2023-02-12 04:02:41,543 INFO [M:0;jenkins-hbase20:41569] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.56 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/05d16c89371645e09edc6fb2cabe8957 2023-02-12 04:02:41,546 INFO [M:0;jenkins-hbase20:41569] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 05d16c89371645e09edc6fb2cabe8957 2023-02-12 04:02:41,547 DEBUG [M:0;jenkins-hbase20:41569] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/05d16c89371645e09edc6fb2cabe8957 as hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/05d16c89371645e09edc6fb2cabe8957 2023-02-12 04:02:41,550 INFO [M:0;jenkins-hbase20:41569] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 05d16c89371645e09edc6fb2cabe8957 2023-02-12 04:02:41,550 INFO [M:0;jenkins-hbase20:41569] regionserver.HStore(1080): Added hdfs://localhost.localdomain:37867/user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/05d16c89371645e09edc6fb2cabe8957, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-12 04:02:41,550 INFO [M:0;jenkins-hbase20:41569] regionserver.HRegion(2947): Finished flush of dataSize ~608.56 KB/623167, heapSize ~719.52 KB/736792, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 22ms, sequenceid=1282, compaction requested=false 2023-02-12 04:02:41,551 INFO [M:0;jenkins-hbase20:41569] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 04:02:41,551 DEBUG [M:0;jenkins-hbase20:41569] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676174561528Disabling compacts and flushes for region at 1676174561528Disabling writes for close at 1676174561528Obtaining lock to block concurrent updates at 1676174561528Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676174561528Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=623167, getHeapSize=736792, getOffHeapSize=0, getCellsCount=1499 at 1676174561529 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676174561529Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676174561529Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676174561534 (+5 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676174561534Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676174561547 (+13 ms)Finished flush of dataSize ~608.56 KB/623167, heapSize ~719.52 KB/736792, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 22ms, sequenceid=1282, compaction requested=false at 1676174561550 (+3 ms)Writing region close event to WAL at 1676174561551 (+1 ms)Closed at 1676174561551 2023-02-12 04:02:41,554 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/11e9db5e-eade-c625-9b84-c7550b462ed2/MasterData/WALs/jenkins-hbase20.apache.org,41569,1676174448151/jenkins-hbase20.apache.org%2C41569%2C1676174448151.1676174449673 not finished, retry = 0 2023-02-12 04:02:41,624 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-12 04:02:41,624 INFO [RS:1;jenkins-hbase20:43977] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,43977,1676174449184; zookeeper connection closed. 2023-02-12 04:02:41,624 ERROR [Listener at localhost.localdomain/35207-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3065e376 rejected from java.util.concurrent.ThreadPoolExecutor@7b579956[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 04:02:41,624 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): regionserver:43977-0x10154d814950002, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-12 04:02:41,624 ERROR [Listener at localhost.localdomain/35207-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@487b2188 rejected from java.util.concurrent.ThreadPoolExecutor@7b579956[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 04:02:41,624 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3b14b715] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3b14b715 2023-02-12 04:02:41,624 INFO [Listener at localhost.localdomain/35207] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-12 04:02:41,655 INFO [M:0;jenkins-hbase20:41569] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-12 04:02:41,655 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-12 04:02:41,655 INFO [M:0;jenkins-hbase20:41569] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:41569 2023-02-12 04:02:41,657 DEBUG [M:0;jenkins-hbase20:41569] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase20.apache.org,41569,1676174448151 already deleted, retry=false 2023-02-12 04:02:41,758 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-12 04:02:41,758 INFO [M:0;jenkins-hbase20:41569] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,41569,1676174448151; zookeeper connection closed. 2023-02-12 04:02:41,758 ERROR [Listener at localhost.localdomain/35207-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2b60ab3b rejected from java.util.concurrent.ThreadPoolExecutor@67050c21[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 04:02:41,758 DEBUG [Listener at localhost.localdomain/35207-EventThread] zookeeper.ZKWatcher(600): master:41569-0x10154d814950000, quorum=127.0.0.1:56679, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-12 04:02:41,758 ERROR [Listener at localhost.localdomain/35207-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1dbdfc88 rejected from java.util.concurrent.ThreadPoolExecutor@67050c21[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 04:02:41,761 WARN [Listener at localhost.localdomain/35207] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-12 04:02:41,768 INFO [Listener at localhost.localdomain/35207] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-12 04:02:41,874 WARN [BP-664456210-148.251.75.209-1676174445046 heartbeating to localhost.localdomain/127.0.0.1:37867] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-664456210-148.251.75.209-1676174445046 (Datanode Uuid 484548f8-2ed9-47b7-81ed-a4821ba20939) service to localhost.localdomain/127.0.0.1:37867 2023-02-12 04:02:41,880 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd/dfs/data/data5/current/BP-664456210-148.251.75.209-1676174445046] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-12 04:02:41,881 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd/dfs/data/data6/current/BP-664456210-148.251.75.209-1676174445046] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-12 04:02:41,882 WARN [Listener at localhost.localdomain/35207] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-12 04:02:41,884 INFO [Listener at localhost.localdomain/35207] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-12 04:02:41,896 WARN [BP-664456210-148.251.75.209-1676174445046 heartbeating to localhost.localdomain/127.0.0.1:37867] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-664456210-148.251.75.209-1676174445046 (Datanode Uuid 6bee6fa0-f002-488e-898d-32eb09388f7e) service to localhost.localdomain/127.0.0.1:37867 2023-02-12 04:02:41,897 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd/dfs/data/data3/current/BP-664456210-148.251.75.209-1676174445046] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-12 04:02:41,897 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd/dfs/data/data4/current/BP-664456210-148.251.75.209-1676174445046] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-12 04:02:41,997 WARN [Listener at localhost.localdomain/35207] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-12 04:02:42,002 INFO [Listener at localhost.localdomain/35207] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-12 04:02:42,112 WARN [BP-664456210-148.251.75.209-1676174445046 heartbeating to localhost.localdomain/127.0.0.1:37867] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-664456210-148.251.75.209-1676174445046 (Datanode Uuid ba999208-985a-462e-b295-ddee73c8b42c) service to localhost.localdomain/127.0.0.1:37867 2023-02-12 04:02:42,115 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd/dfs/data/data1/current/BP-664456210-148.251.75.209-1676174445046] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-12 04:02:42,116 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/d979d5e4-99bb-c079-e146-1d58414c4ab8/cluster_6a80d0bd-159a-626f-f74c-fa85164f71bd/dfs/data/data2/current/BP-664456210-148.251.75.209-1676174445046] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-12 04:02:42,140 INFO [Listener at localhost.localdomain/35207] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-02-12 04:02:42,322 INFO [Listener at localhost.localdomain/35207] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-12 04:02:42,542 INFO [Listener at localhost.localdomain/35207] hbase.HBaseTestingUtility(1293): Minicluster is down